-- Logs begin at Tue 2022-09-13 19:14:28 UTC, end at Tue 2022-09-13 20:46:57 UTC. -- Sep 13 19:43:04.595752 dv-cinder-854274-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 19:43:04.604926 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 19:43:04 2022] *** Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: nodename: dv-cinder-854274-5-iscsi Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: machine: x86_64 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: clock source: unix Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: pcre jit disabled Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: detected number of CPU cores: 4 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: current working directory: / Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: detected binary path: /usr/bin/uwsgi-core Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: your processes number limit is 47680 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: your memory page size is 4096 bytes Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: detected max file descriptor number: 2048 Sep 13 19:43:04.607002 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: lock engine: pthread robust mutexes Sep 13 19:43:04.608689 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: thunder lock: enabled Sep 13 19:43:04.608689 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 19:43:04.608689 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 19:43:04.647827 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: Python main interpreter initialized at 0x557d572119c0 Sep 13 19:43:04.647827 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: python threads support enabled Sep 13 19:43:04.647827 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: your server socket listen backlog is limited to 100 connections Sep 13 19:43:04.647827 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: your mercy for graceful operations on workers is 90 seconds Sep 13 19:43:04.648259 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 19:43:04.648259 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: *** Operational MODE: preforking *** Sep 13 19:43:04.648605 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: *** uWSGI is running in multiple interpreter mode *** Sep 13 19:43:04.648605 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: spawned uWSGI master process (pid: 81563) Sep 13 19:43:04.648797 dv-cinder-854274-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 19:43:04.651153 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 1 (pid: 81575, cores: 1) Sep 13 19:43:04.651153 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: spawned uWSGI worker 2 (pid: 81576, cores: 1) Sep 13 19:43:04.651153 dv-cinder-854274-5-iscsi devstack@keystone.service[81563]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 19:43:08.943823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:43:08.943823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: from cryptography.utils import int_from_bytes Sep 13 19:43:08.944661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:43:08.944661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: from cryptography.utils import int_from_bytes Sep 13 19:43:08.944661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:43:08.944661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: from cryptography.utils import int_from_bytes Sep 13 19:43:08.945255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:43:08.945255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: from cryptography.utils import int_from_bytes Sep 13 19:43:08.995546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:43:08.995788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:43:08.995788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:43:08.995992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:43:08.996061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] command line args: [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:43:08.996119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] command line args: [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:43:08.996180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:43:08.996236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:43:08.996296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ================================================================================ {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:43:08.996363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ================================================================================ {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:43:08.996558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] admin_token = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.996684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] admin_token = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.996798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] config_dir = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.996867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] config_dir = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.996961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] config_source = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] config_source = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] debug = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] debug = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.997972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.998970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_config_append = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_config_append = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_dir = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_dir = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:08.999981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_file = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_file = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.000915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.001978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.002694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.003282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.003418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.003531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.003722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.003829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] publish_errors = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] publish_errors = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.004909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] standard_threads = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] standard_threads = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.005921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] use_journal = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] use_journal = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] use_json = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.006931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] use_json = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] use_stderr = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] use_stderr = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] use_syslog = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] use_syslog = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:43:09.007865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.007973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.008982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.009903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.010960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.011897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.012983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.013905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.014920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.015956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.016982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.017975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.018916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.019935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.020962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.021952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.022932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.023913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.024936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.025994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.026940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.027998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.028959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.029986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.030863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.031910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.032909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.033944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.034960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.external = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.035927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.password = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.external = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] auth.token = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.036966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.password = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] auth.token = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.037876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.038993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.039943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.040995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.041917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.042906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.043968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.044951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.045881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.046932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.047977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.048896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.049987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.050890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.051966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.052903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.053939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.054914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.055898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.056978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.057924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.058952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.059895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.060946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.061914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.062973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.063912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.064971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.065908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.066891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] role.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] role.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.067945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.068944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.069961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.070891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] role.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] role.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.071952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.072926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.073950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.074979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.075990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.076820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.077894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.connection = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.078961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.079906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.080939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.081999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.082985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.connection = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.083099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.084877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.085102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.085486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.085766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.086003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.086238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.086609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.086897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:43:09.087072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:43:09.110086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.110306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.110554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.112016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.112294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.112452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.119557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.121306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.149076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.149380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.150535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.150698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.207867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.208061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.257876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.258159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.258260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.258362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.419169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.419406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.419550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.419802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.419948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.420839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.443759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.443995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.444964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.445080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.445284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.446064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.446064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.446381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.456967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.460581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.470096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.470376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.470596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.470841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.471897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.472125 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.472347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.472566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.472793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:43:09.488376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.488583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:43:09.488737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.488857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:43:09.488978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:43:09.489224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:43:09.489486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.489992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.490131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.490258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.490405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.500289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.500481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.500611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.500757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.500885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.501011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.501135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.501301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.508212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.508411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.511229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.511380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.511514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.511713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.511859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.511998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.512957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.513911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.514044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.514177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.537026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.537261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.540818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.541081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.541252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.541418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.543695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.543861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.544010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.544124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.544244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.544361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.550750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.551008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.551149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.551272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.557022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.557259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.557383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.557574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.557696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.557866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.557983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.558869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.559004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.559115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.559231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.574906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.575091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.575327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.575478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.579343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.579596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.582668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.582944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.583108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.583243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.587206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.587481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.590407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.590638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.593711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.593840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.594904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.595028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.595184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.595302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.610973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.611143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.611264 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.611417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.611531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.611686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.611799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.612017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.618932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.619223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.620458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.620777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.621963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.622244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.622359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.622603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.625099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.625367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.625525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.625657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.625780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.625908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.626051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.626184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.626308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.626623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.637507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.637792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.637909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.638064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.638180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.638351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.638499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.638638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.638751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.639035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.639170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.639299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.639438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.639592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.654885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.655193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.657910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.658215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.660854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.661152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.662344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.662626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.663836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.664118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.664236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.664400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.664515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.664801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.670908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.671282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.672392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.672742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.675332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.675681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.675767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.675988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.676122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.676278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.676363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.676607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.684143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.684451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.684535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.684724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.684849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.685008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.691752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.692039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:43:09.692307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.692953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.693946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.694912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.695036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.695154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.695315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.695397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.695601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:43:09.729673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557d572119c0 pid: 81575 (default app) Sep 13 19:43:09.729937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557d572119c0 pid: 81576 (default app) Sep 13 19:43:09.733200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f153-9af3-4c80-92bc-18672593ab91 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:09.733543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f153-9af3-4c80-92bc-18672593ab91 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:09.733821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495f153-9af3-4c80-92bc-18672593ab91 None None] PATH_INFO: `/v3` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:09.738049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Tue Sep 13 19:43:09 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:16.874142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:16.874823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:16.874823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:16.879105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:16.884724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:16.885439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:16.885439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:16.955734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: INFO dbcounter [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] Registered counter for database keystone Sep 13 19:43:16.975599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG oslo_db.sqlalchemy.engines [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81575) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:43:16.983675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writer thread running {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:43:17.166652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG passlib.handlers.bcrypt [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81575) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:43:17.166942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG passlib.handlers.bcrypt [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81575) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:43:17.225512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:17.227209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:17.359910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:43:17.874224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:17.943973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: INFO dbcounter [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] Registered counter for database keystone Sep 13 19:43:17.966116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG oslo_db.sqlalchemy.engines [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81576) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:43:17.974726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writer thread running {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:43:18.094089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635824f-b5f1-43cc-95e5-41891225ccb7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:18.098702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:18.099364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.099640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.099908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.108959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:18.160622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING keystone.common.rbac_enforcer.enforcer [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:43:18.160836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c31145-d292-4bc1-a591-a7f8c2629096 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:18.171531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Sep 13 19:43:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:43:18.177953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:18.197159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d79c2-7820-4594-954d-e0195e5d2909 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:18.203053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:18.204230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.204513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.204787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] PATH_INFO: `/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.206523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] RBAC: Authorizing `identity:get_project(project_id=02e765af7d454a8f9fe205c67fe1fdbb)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:18.259102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:43:18.259440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45ca93-cd40-4ca2-8e36-45cc31b06f6b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:18.264369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Tue Sep 13 19:43:18 2022] GET /identity/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:20.415182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe03276-5997-455e-b91c-4874d3abd2ec None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.415777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe03276-5997-455e-b91c-4874d3abd2ec None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.415777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe03276-5997-455e-b91c-4874d3abd2ec None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.420404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:20.427008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.427541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.427541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.450215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:20.451855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.460968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:43:20.728586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe03276-5997-455e-b91c-4874d3abd2ec None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:20.741586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe03276-5997-455e-b91c-4874d3abd2ec None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:20.746657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:20.747303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.747584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.747856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.757060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:20.758966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b2f4-84a2-4878-9ad2-625ce2d7114f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:20.769363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Sep 13 19:43:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:43:20.775909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:20.786124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd99435-b06c-4390-9e75-6fc0e8f8f40a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:20.791569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:20.792269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.792546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.792812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.794274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:20.796187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a0f83-88cf-43a5-9707-2ef940b1d1bb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:20.797831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Tue Sep 13 19:43:20 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:23.031913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f977-7019-4d8b-b3a3-afeeb2ce94c5 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.032480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f977-7019-4d8b-b3a3-afeeb2ce94c5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.032480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441f977-7019-4d8b-b3a3-afeeb2ce94c5 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.033859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:23.043831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.044724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.044724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.071457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:23.073234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.161109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab13e7-75a9-4a72-9a23-3c012fc23ad8 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.161417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab13e7-75a9-4a72-9a23-3c012fc23ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.161652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab13e7-75a9-4a72-9a23-3c012fc23ad8 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.163039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:23.171267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.171546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.171825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.178533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:23.215017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG passlib.handlers.bcrypt [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81576) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:43:23.215394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG passlib.handlers.bcrypt [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81576) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:43:23.264512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:23.266223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.374359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:23.461535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.493909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.600444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5f295-adf6-456a-85cc-9b28b2d2b0e5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:23.637625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:23.638195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.638511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.638792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.644881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.651576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:23.660807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:23.662548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:23.664005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:23.668104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.765303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b09bd-3d7e-4ea8-94d1-2f230b874426 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:23.767778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.775208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.798011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04700bfb-ec28-4591-8a43-354ff8afc252 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:23.822214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:23.823763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.824239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.824707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.833741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:23.834402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.834672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.834942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.835627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:23.835929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:23.837337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:23.850490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:23.856580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.918417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:23.919923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:23.988745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e475f-c9d2-4933-86de-ffd5b445f0e2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.027546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.028212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.028490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.028764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.029977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.031600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.033275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 9/16] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:24.039206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:24.056223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:24.056928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 8/17] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:43:24.063311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:24.066690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:24.171485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce2ba0-1b25-4b00-826d-d0d00e8e56b7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.180209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4560a-3b3a-482a-8153-9304d142e7eb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.209334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.209999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.210273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.210572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.211785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.213324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.215000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:24.221173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:24.230738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.231606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.231867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.232139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.252755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.254274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:43:24.275867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:24.277925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:43:24.279823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:43:24.279823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:43:24.279823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:24.279823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:24.290901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:24.319274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:24.342468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098697ec-8358-4ca7-ae07-c939c8db63da None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.373233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:24.380858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.381491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.381770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.382042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.382813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.domain.id=default, user.id=0f81252e23c341beb35dc2c378ed375c) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:24.383122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.384658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ba84-5d92-4da9-a6ca-85c270d31d0c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.391933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Sep 13 19:43:24 2022] GET /identity/v3/role_assignments?user.id=0f81252e23c341beb35dc2c378ed375c&scope.domain.id=default&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 233 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:24.441573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a95f7-ff83-4b44-abf4-814de49b417d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.498155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.498834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.499169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.499454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.500136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:24.500433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.501874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ed6d2-941c-4e0b-88c7-cd9789a72aef None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.522016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:27.532656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acee4e-4d77-42fb-8b85-aad248552e86 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.533474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acee4e-4d77-42fb-8b85-aad248552e86 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.533474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acee4e-4d77-42fb-8b85-aad248552e86 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.534793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:27.542247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.542247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.542247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.571912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:27.573149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.661605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22950-d1b8-40da-9347-1e0116b0e315 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.662020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22950-d1b8-40da-9347-1e0116b0e315 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.662160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d22950-d1b8-40da-9347-1e0116b0e315 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.663558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:27.673454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:27.675209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45029b-ee6b-4e52-9c16-c06a6f4354f5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.675496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45029b-ee6b-4e52-9c16-c06a6f4354f5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.675774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45029b-ee6b-4e52-9c16-c06a6f4354f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.706398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-1c45029b-ee6b-4e52-9c16-c06a6f4354f5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:27.708949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1c45029b-ee6b-4e52-9c16-c06a6f4354f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.879639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:28.302103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:28.399929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:28.516134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81912b-5d16-497f-8774-af53ffbff59f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:28.595130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:28.596098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.596098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.596355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.596963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:28.598522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9337a-76e3-4fc1-9da0-b0601e010329 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:28.721823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268480-854f-4403-9069-44493137bc0b None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.723329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268480-854f-4403-9069-44493137bc0b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.724031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268480-854f-4403-9069-44493137bc0b None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.726778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 15/26] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:28.736357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe94ad3-c2d6-4a79-8ddf-93dc06d632fe None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.737223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe94ad3-c2d6-4a79-8ddf-93dc06d632fe None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.737930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe94ad3-c2d6-4a79-8ddf-93dc06d632fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.769014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2fe94ad3-c2d6-4a79-8ddf-93dc06d632fe None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:28.771668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2fe94ad3-c2d6-4a79-8ddf-93dc06d632fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:28.839620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 12/27] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Sep 13 19:43:28 2022] POST /identity/v3/roles => generated 219 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:28.844097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fec49-235e-4396-b743-61511f39b733 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.845005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fec49-235e-4396-b743-61511f39b733 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.845717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fec49-235e-4396-b743-61511f39b733 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.848435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 13/28] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:28.850276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fec49-235e-4396-b743-61511f39b733 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:28.914404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 16/29] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:28.914404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9789834-1b5c-46da-81b5-0f23cb0fe53c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.914404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9789834-1b5c-46da-81b5-0f23cb0fe53c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.914404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9789834-1b5c-46da-81b5-0f23cb0fe53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.928754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:28.959106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e9789834-1b5c-46da-81b5-0f23cb0fe53c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:28.961493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e9789834-1b5c-46da-81b5-0f23cb0fe53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:29.107557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 17/30] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:29.108917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0a9cd-822f-438f-86fd-22a61caa0ce4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.109276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0a9cd-822f-438f-86fd-22a61caa0ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.109552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0a9cd-822f-438f-86fd-22a61caa0ce4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.111135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 18/31] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:29.117452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.117619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fec49-235e-4396-b743-61511f39b733 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.117717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.118025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.129075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:29.151973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:29.153213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:29.172964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.173714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.173994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.174268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.186772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:29.188520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:43:29.205341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:29.206241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:43:29.207710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:43:29.207710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:43:29.207710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:29.207710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 14/32] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:29.212987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.290853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 19/33] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:29.369626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b273e4-a4ae-433d-8ea5-35c595a99462 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.382514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.407169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.407809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.408097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.408378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.409043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:29.409354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:29.410883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:29.424331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 15/34] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:29.431134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.464062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:29.502009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:29.554241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f96cb-9897-423b-adea-a09a509cf4f6 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.563851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967c95c-a627-4923-b416-5698048f5055 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.600773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.601734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.602295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.602625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.602940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.603042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.603435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.603538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.604537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:29.606350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:29.615640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:29.621400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:29.622232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:29.623036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:29.627968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 20/35] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:29.628992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.642293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 16/36] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:43:29 2022] POST /identity/v3/projects => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:43:29.643593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.729901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:29.776548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99897da6-f662-463b-ba75-1c8b6831c2e7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.793071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6594b-0be3-4f43-94a2-d2b5e77b084f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:29.820388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:29.839933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.840179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:29.840596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.840881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.841024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:29.841150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.841292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:29.841619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:29.841848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:29.842502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:29.842970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:29.843451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:29.844548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:29.866933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 17/37] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:29.868070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:29.979065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 21/38] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Sep 13 19:43:29 2022] POST /identity/v3/roles => generated 225 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:29.980126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:30.013311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e2779-427f-4aaa-945a-bc0322fe12b7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:30.060606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:30.069434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:30.070467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.071126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.071427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.073483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:30.075963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:30.078853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 18/39] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:30.085630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:30.168361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3374245-38d3-4196-9afa-f5929ebdf616 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:30.223400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:30.224267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.224694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548e36-1806-4528-a5f6-46469ebdd209 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:30.224812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.225248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.227220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:30.228888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c26cf1-baec-4587-b843-7a62942879a8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:30.230977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 22/40] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:29 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:30.231970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda3f8-b065-4a00-a0ab-1f41df6c6fcd None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.233367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda3f8-b065-4a00-a0ab-1f41df6c6fcd None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.233732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eda3f8-b065-4a00-a0ab-1f41df6c6fcd None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.240035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 23/41] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:30.241113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eda3f8-b065-4a00-a0ab-1f41df6c6fcd None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:30.287426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:30.288547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.288859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.289160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.290364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:30.292154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255e274-2b16-4936-bb15-dd61903d1e0f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:30.326457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 19/42] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:43:30 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:43:30.327916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.328209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.328484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.351998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:30.353768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:30.415945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:30.432489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:30.457326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 20/43] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:30.517988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eda3f8-b065-4a00-a0ab-1f41df6c6fcd None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:30.579968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:30.580903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.581388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.581726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.583817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:30.584967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:30.588528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 24/44] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:30.594124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:30.738168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a99d7-898e-4691-a6e5-dfbb83d191f2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:30.775074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:30.775657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.775938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.776212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] PATH_INFO: `/v3/domains/default/users/0f81252e23c341beb35dc2c378ed375c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.778503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0f81252e23c341beb35dc2c378ed375c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:30.779976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6804f8-7bc6-4bf3-8073-db3bd0562691 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:30.806428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 21/45] 10.222.0.52 () {58 vars in 1361 bytes} [Tue Sep 13 19:43:30 2022] PUT /identity/v3/domains/default/users/0f81252e23c341beb35dc2c378ed375c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:43:30.840686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:31.032914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9732fe9-ec49-4470-a356-347ce834db74 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:31.059687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:31.075276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:31.075917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:31.076197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:31.076466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:31.077157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:31.078694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e2dc6-735e-41da-9ddc-c75b4227eeb6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:31.096033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 25/46] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Sep 13 19:43:30 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:32.582321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eed89-3f44-44cb-b09f-5f447b67b184 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:32.582804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eed89-3f44-44cb-b09f-5f447b67b184 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:32.582921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eed89-3f44-44cb-b09f-5f447b67b184 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:32.584310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 22/47] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:32.589767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2d19d-7e5c-4a9f-9091-c3bbb2202ffa None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:32.590279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2d19d-7e5c-4a9f-9091-c3bbb2202ffa None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:32.590453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2d19d-7e5c-4a9f-9091-c3bbb2202ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:32.610157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cbf2d19d-7e5c-4a9f-9091-c3bbb2202ffa None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:32.611708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cbf2d19d-7e5c-4a9f-9091-c3bbb2202ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:32.686740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 26/48] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:32.950992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eed89-3f44-44cb-b09f-5f447b67b184 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:33.009245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:33.048762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933ab-43cf-46f0-89e2-7fe17f93881b None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.049064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933ab-43cf-46f0-89e2-7fe17f93881b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.049350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933ab-43cf-46f0-89e2-7fe17f93881b None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.050771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 27/49] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:33.057742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.058254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.059074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.077830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eed89-3f44-44cb-b09f-5f447b67b184 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:33.079949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:33.081162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:33.123907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:33.124598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.124891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.125160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.126348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:33.127848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:33.129449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:33.135105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:33.155307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 28/51] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:33.245419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9cca5-67fd-44cd-bf8f-d1d62df47427 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:33.256892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:33.285733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:33.286440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.286730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.287008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.287713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:33.289301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:33.321746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 24/52] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Sep 13 19:43:33 2022] POST /identity/v3/projects => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:33.424409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:33.477010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:33.547852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7929495-f071-48b4-8cc9-839e2d5398d9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:33.584473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:33.585134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.585417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.585688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.597136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:33.602143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:33.603766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:33.604484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 29/53] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:33.609782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:33.844163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c45fd8-007a-4a67-9476-0c4fe97471c5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:33.880888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:33.881525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:33.881805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:33.882083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:33.882803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:33.883110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:33.884559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:33.906570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 25/54] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:33.913728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:34.041546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:34.044784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ced781-f3a9-45b4-bc3e-74e7e9afdb86 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:34.079734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:34.080412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:34.080694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:34.080995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:34.082200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:34.083705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:34.085307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 30/55] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:33 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:34.091151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:34.124399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:34.212053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a2d6e-c368-4d0d-8556-5f6a1930ec61 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:34.253959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:34.254804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:34.255143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:34.255464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:34.256703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:34.258206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86174e15-7364-4f48-8f5e-e1816379f6af None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:34.260014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 26/56] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:34.266389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:34.372108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d61b6-26a7-4560-bdba-e32520c5f244 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:34.407134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:34.407910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:34.408226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:34.408485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:34.409205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.domain.id=default, user.id=0f81252e23c341beb35dc2c378ed375c) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:34.409508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:34.411029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4aa554-2cc2-4423-9d4d-b954a4427915 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:34.417126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 31/57] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Sep 13 19:43:34 2022] GET /identity/v3/role_assignments?user.id=0f81252e23c341beb35dc2c378ed375c&scope.domain.id=default&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:35.544240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b3c3-124a-43f3-a545-675a6f14b401 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:35.545021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b3c3-124a-43f3-a545-675a6f14b401 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:35.545166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b3c3-124a-43f3-a545-675a6f14b401 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:35.546742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 27/58] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:35.552481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:35.552890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:35.553037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:35.573039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:35.574296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:35.601362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:35.644383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 32/59] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:35.910008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b3c3-124a-43f3-a545-675a6f14b401 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:36.012179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:36.022855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b3c3-124a-43f3-a545-675a6f14b401 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:36.058091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:36.058835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:36.059174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:36.059515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:36.060726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:36.062274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224fca6-b3f9-4ad7-ac83-ca90bcb49c71 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:36.063931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 28/60] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:36.070686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:36.178699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035d55d-42f5-4cc4-9ec4-b34491e9adee None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:36.224570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:36.225496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:36.225777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:36.226044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:36.226742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:36.228249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdee5f8-d317-43b6-9d03-fdf77fb4a0b9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:36.261041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 33/61] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:43:36 2022] POST /identity/v3/users => generated 275 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:39.722635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dad44-16ae-4366-b6bb-2c42687c5a3e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.723149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dad44-16ae-4366-b6bb-2c42687c5a3e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.723576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618dad44-16ae-4366-b6bb-2c42687c5a3e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.725922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 29/62] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:39.732027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17491743-50e8-409f-9be5-a2601832232b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.732529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17491743-50e8-409f-9be5-a2601832232b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.732602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17491743-50e8-409f-9be5-a2601832232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.759606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-17491743-50e8-409f-9be5-a2601832232b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:39.761269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-17491743-50e8-409f-9be5-a2601832232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:39.777087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:39.872278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 34/63] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:39.952895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314019b1-c086-4656-a944-a4dec2fddb0a None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.953528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314019b1-c086-4656-a944-a4dec2fddb0a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.953990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314019b1-c086-4656-a944-a4dec2fddb0a None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.955612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 30/64] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:39.964650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1a8f-9d5e-4cee-8170-97f2fb4d5b3f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.965307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1a8f-9d5e-4cee-8170-97f2fb4d5b3f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.965781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1a8f-9d5e-4cee-8170-97f2fb4d5b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.995718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-961b1a8f-9d5e-4cee-8170-97f2fb4d5b3f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:39.997341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-961b1a8f-9d5e-4cee-8170-97f2fb4d5b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.009714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f2c4c-4a49-49b4-ad03-7d430770ab79 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.010710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f2c4c-4a49-49b4-ad03-7d430770ab79 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.011185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f2c4c-4a49-49b4-ad03-7d430770ab79 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.012838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 31/65] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:40.019075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.019696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.020197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.050463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:40.053539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.117562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 35/66] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:40.119763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc5e53-dac8-4a78-be84-2f06553e8ec5 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.120287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc5e53-dac8-4a78-be84-2f06553e8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.120972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc5e53-dac8-4a78-be84-2f06553e8ec5 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.123468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 36/67] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:40.134141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.135092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.135920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.158251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:40.159896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.220931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 32/68] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:40.256394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 37/69] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:40.321630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.410414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.424980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.434785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.443305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95db73-65b6-4b99-8258-70abea3d66dd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:40.490920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:40.492441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.493075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.494012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.510090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-9c030d8c-5f65-4359-87e9-c4f360c196cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:40.520734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:40.522118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:40.523623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:40.525467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 33/70] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:40.525761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdeb4-dd5e-497e-83c5-164ec50ec669 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.526022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdeb4-dd5e-497e-83c5-164ec50ec669 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.526296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecdeb4-dd5e-497e-83c5-164ec50ec669 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.527776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 34/71] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:40.528318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecdeb4-dd5e-497e-83c5-164ec50ec669 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.546473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599ae26-a43e-4058-9dbb-37dea8e6b369 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:40.596556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:40.597277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.597563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.597833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.609335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:40.614256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:40.615118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:40.615844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 38/72] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:40.616442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.671509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecdeb4-dd5e-497e-83c5-164ec50ec669 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:40.691197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.709171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:40.709824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.710106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.710419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.722837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-b3ea0880-f454-4cd3-892e-8ec1524b881f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:40.727789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:40.728632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:40.729515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 35/73] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:40.730887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.730887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.730887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.734733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034d84e-7488-4a7e-beb8-2b6bc57e31bb None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:40.749486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:40.750754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.753890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.776393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:40.777063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.777351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.777635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.778318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:40.778666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:40.780115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:40.789279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 39/74] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:40.790203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.842118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 36/75] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:40.843042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:40.912825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cfe94-ee89-4a94-ab13-04af8ff17434 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:40.947505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.957715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:40.958402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.958747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.958991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.971160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:40.976717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:40.977664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:40.978597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 40/76] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:40.979279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.004632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0fe67-9023-449c-8885-a3f8a7d5ae6d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.042547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.043427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.043850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.044248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.045182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:41.045485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.047032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.060934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.060934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.060934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.109872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a807c-294a-41e7-b955-a0208d90f442 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.145114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.145742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.146028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.146301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.147010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:41.147314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.148746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.156931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:43:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.157782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.182510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd33ba-c6db-46c4-a2dd-fb5d494f086e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.215859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.216514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.216793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.217066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.219585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.234283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.235858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.237641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.238503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.272588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec6fd3-0d40-447a-9559-a7fbd09a70a1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.307597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.308240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.308643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.308773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.309430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:41.309728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.311160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.315642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.320651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 42/80] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.321608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.359008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35b504-26e7-47cb-b1fd-84e922d5de65 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.395534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.396282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.396569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.396881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.398025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.399554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.401192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 39/81] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.401967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.429641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f590af-6b4d-4932-bd6e-50b347c4b50e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.444245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.465561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.466256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.466546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.466789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.467951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.469464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.471112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 43/82] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.471875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.510919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d5424-b881-4141-9d97-be4b9216cb7c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.537779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.547890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.548536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.548811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.549078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.550293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.551705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.553455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 40/83] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.554232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.582950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4407d1-4c5e-47da-ab80-b43b548683e0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.640203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.640836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.641120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.641396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.665572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c5f07-0394-4801-8832-2b8fa9263951 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.690360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.699904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.700546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.700823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.701094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.702226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.703724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.705341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 41/84] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:41.706109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.713837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.715384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.768134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.808668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130ffac-bfa2-4df5-bc7f-e12c441211b4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.815738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 44/85] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:41.816615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.845878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.846545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.846830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.847108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.899056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.914819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.916338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.918605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8340e44-a8d8-49fc-988b-6d43f906769e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.953192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.953816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.954096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.954378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.955101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.956818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.977900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 45/86] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:43:41 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:41.978788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.981296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.034510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 42/87] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:42.035579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.109455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2817e-deb7-460f-8271-2875baf2c5f9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.147983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.148641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.149208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.149208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.155878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea2d81-0f80-492a-b650-bbe89fa8cb5c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.158799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.203479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.204226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.204576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.204925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.230242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.232025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.241492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.296702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.298132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.370750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 46/88] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:42.371754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.408333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 43/89] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:42.409523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.423794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.473857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.481841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bb5eb-e6fb-4cd4-8d9a-fb345293e272 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.515148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9c88-e0b7-4835-80b5-2b648299ccf3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.526924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.528036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.528334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.528611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.530114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.533237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.535465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 47/90] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:42.539212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.566660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.567729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.568832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.569338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.571159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.573630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.576266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:42.577955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.669977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b6f57-02af-47d2-8e00-580d1fb6ec52 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.689157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.722203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.723036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.723381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.723761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.725363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.726991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.728944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 48/92] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:42.730197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.732072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da5df6-6924-43c3-885b-d6836bd46356 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.768012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:42.776483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.777248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.777453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.777732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] PATH_INFO: `/v3/projects/1cc9adf0ab9c4142bd223552de529932` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.779430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] RBAC: Authorizing `identity:get_project(project_id=1cc9adf0ab9c4142bd223552de529932)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.781452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.783577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 45/93] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/projects/1cc9adf0ab9c4142bd223552de529932 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:42.784763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.879718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837f65a-fc16-4f73-a18c-11be1ca43214 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.909577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f6f8a-f327-40ac-8d0c-206832626317 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:42.922990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.924135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.924700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.925170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.926306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=a418650541634431838ac8d10049c192) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:42.926909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.928634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:42.935118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 49/94] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:42.936335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:42.992883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:42.993885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:42.994169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:42.994475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:42.995234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=0f81252e23c341beb35dc2c378ed375c) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:42.995570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:42.997588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:43.002877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:43.018101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/role_assignments?user.id=0f81252e23c341beb35dc2c378ed375c&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:43.018101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:43.152967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9735a1-a099-4a65-a743-2c17881ff365 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:43.189777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:43.255734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:43.255734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:43.255734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:43.255734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:43.256210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=a418650541634431838ac8d10049c192) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:43.256210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:43.257291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f158b7d-d005-45e9-bc98-5807d2c577f7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:43.270696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:42 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=f12fcc11a666405fa05a294664d03492 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:43.274910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed15329-5236-4923-b969-fcf06f947c27 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:43.354406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:43.355065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:43.355376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:43.355642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:43.356445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=1cc9adf0ab9c4142bd223552de529932, user.id=a418650541634431838ac8d10049c192) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:43.356772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:43.358610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c97b-7081-4411-9225-dcb35dbd01fe None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:43.372317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:43 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=1cc9adf0ab9c4142bd223552de529932&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:44.704067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0b9bb-ce31-4b5a-9e4c-5bcd08048352 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:44.704589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0b9bb-ce31-4b5a-9e4c-5bcd08048352 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:44.704589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0b9bb-ce31-4b5a-9e4c-5bcd08048352 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:44.706161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:44.712172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a0cf-31be-4fba-8341-858a5acef52b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:44.712803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a0cf-31be-4fba-8341-858a5acef52b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:44.712803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a0cf-31be-4fba-8341-858a5acef52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:44.738008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4b90a0cf-31be-4fba-8341-858a5acef52b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:44.739288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4b90a0cf-31be-4fba-8341-858a5acef52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:44.825182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 48/99] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:45.388160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0b9bb-ce31-4b5a-9e4c-5bcd08048352 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:45.542910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d69e-831d-477d-bebb-076af138a314 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.543294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d69e-831d-477d-bebb-076af138a314 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.543490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9d69e-831d-477d-bebb-076af138a314 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.545865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 49/100] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:45.555127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddea79-85e5-4129-a722-4db115feafde None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.555700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddea79-85e5-4129-a722-4db115feafde None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.556177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddea79-85e5-4129-a722-4db115feafde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.583021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ddddea79-85e5-4129-a722-4db115feafde None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:45.586442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ddddea79-85e5-4129-a722-4db115feafde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:45.602457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0b9bb-ce31-4b5a-9e4c-5bcd08048352 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:45.604274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:45.697091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:45.697677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.697957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.698227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.699491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:45.700034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:45.701019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:45.704892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 52/101] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:45.711079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:45.735904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 50/102] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:45.928391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b91d58-c45f-4565-8507-65477161b49f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:45.980032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:45.980685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.980967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.981240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.981920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:45.983540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308afa45-a052-47e2-9062-83a7f62eb22a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:46.033553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 53/103] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:43:45 2022] POST /identity/v3/projects => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:43:46.143720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f331-f4dc-4c10-b6b9-82ea53be0430 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.144415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f331-f4dc-4c10-b6b9-82ea53be0430 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.144899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f331-f4dc-4c10-b6b9-82ea53be0430 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.146584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 51/104] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:46.153875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ad5-2222-48c0-88cf-5d83d1480fe3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.154161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ad5-2222-48c0-88cf-5d83d1480fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.154512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951ad5-2222-48c0-88cf-5d83d1480fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.182939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f331-f4dc-4c10-b6b9-82ea53be0430 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:46.184940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6a951ad5-2222-48c0-88cf-5d83d1480fe3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:46.186866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6a951ad5-2222-48c0-88cf-5d83d1480fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:46.217835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:46.327445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 54/105] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:46.395369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f331-f4dc-4c10-b6b9-82ea53be0430 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:46.453180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:46.453852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.454137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.454434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.469181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:46.476631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:46.477551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:46.478306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:46.478943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181db3-c153-4937-9fcf-0af2780192b6 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.478943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181db3-c153-4937-9fcf-0af2780192b6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.478943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181db3-c153-4937-9fcf-0af2780192b6 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.478943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 55/106] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:46.479179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 52/107] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:46.484858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:46.485532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.486023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.486538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.508387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:46.510634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:46.589856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:46.662877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 56/108] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:46.693968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cca68-93a7-4553-a353-f3c701f6ad1a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:46.741806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:46.742776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.743061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.743769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:46.744514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:46.744824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:46.746240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:46.756668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 53/109] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:46.762607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:46.872943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:46.900539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:46.936581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c7a3-f8dd-4f95-8d18-06e0bd8d1bb3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:46.984757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:46.985449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:46.985766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:46.986085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] PATH_INFO: `/v3/users/44804d7f61b9408bbd25cafbd9aab87d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.002437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] RBAC: Authorizing `identity:get_user(user_id=44804d7f61b9408bbd25cafbd9aab87d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.004280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831edd7-c98f-42dc-a50f-970ea9f5d9cc None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.006115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 57/110] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:46 2022] GET /identity/v3/users/44804d7f61b9408bbd25cafbd9aab87d => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.007405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c4dc2-f687-48b6-872d-79aa24df62b1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.007695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c4dc2-f687-48b6-872d-79aa24df62b1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.007966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c4dc2-f687-48b6-872d-79aa24df62b1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.009366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 58/111] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:47.011670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9e8d-327a-4730-b8de-3f4a22ba1444 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.013820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9e8d-327a-4730-b8de-3f4a22ba1444 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.014117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9e8d-327a-4730-b8de-3f4a22ba1444 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.015494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 59/112] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:47.016210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9e8d-327a-4730-b8de-3f4a22ba1444 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.018875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f622f-53dd-42c6-87f2-901268451153 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.031578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.057575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.058226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.058533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.058805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.069671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:47.075389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:47.076267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:47.077030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 54/113] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:47.077651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.130634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9e8d-327a-4730-b8de-3f4a22ba1444 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.169354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.170062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.170348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.170651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.179238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.180591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39697a4e-c9ae-47a0-9a18-762f51a2a0b5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.183571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-0e01eb23-206d-4094-b12c-45e375c5e36b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:47.188767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:47.189733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:47.190633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 60/114] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:47.191704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df27e-f39c-4246-855a-474aac37473b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.191991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df27e-f39c-4246-855a-474aac37473b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.192267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584df27e-f39c-4246-855a-474aac37473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.211802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-584df27e-f39c-4246-855a-474aac37473b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:47.213117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-584df27e-f39c-4246-855a-474aac37473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.224080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.224707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.225437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.225715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.267664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.284446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 61/115] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:47.291882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.292262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.292467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.306141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.307971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.310938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:47.312198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.383684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 62/116] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:47.391400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.424577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 55/117] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:47.425822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.501776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.503532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.537965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c559-3e5e-4670-ad25-e55820a089cb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.569767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c801d-418f-461f-84ba-1469361a2258 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.578904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.579948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.580452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.580987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.581917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:47.582438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.584081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.594635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 56/118] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.599739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.617901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.619567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.620503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.620977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.622128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:47.622920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.625071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.637981 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 63/119] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.639229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.720751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e7815-e37e-41b0-8a9b-153ab11dedc6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.753922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db8997-6888-4318-b42b-8e6169eddb0c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.761264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.770006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.770587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.770866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.771143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.772787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.774278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.781685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 57/120] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.781685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.782354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.798838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.799511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.799798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.800075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.811770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:47.817170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:47.818025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:47.818819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 64/121] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:47.819433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.893861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f03cbe-3c6f-4431-a330-3e3185b41f10 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.929796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41122689-903a-436f-a9e9-2cb9ea8bb33b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.932365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.933164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.933443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.933731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.934969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.936518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.943000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 58/122] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.943000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.969256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.969924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.970203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.970517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.971703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.973206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.974896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 65/123] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:47.980471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.014893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.034242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.060892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccb673-bc16-4f8d-a6b1-9f06000dfe04 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.095688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315cbff1-31e6-4689-bb37-32f437ea3257 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.098872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.099526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.099808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.100084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.111087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:48.116388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:48.117243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:48.118001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 59/124] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:48.119018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.137162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.138145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.138747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.139207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.140028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=44804d7f61b9408bbd25cafbd9aab87d) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:48.140350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.141888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.158486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 66/125] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/role_assignments?user.id=44804d7f61b9408bbd25cafbd9aab87d&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=3e412976f84c4f64914a78835f1b3566 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.158486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.258949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828809-e4d2-4869-9c71-fafcefc1b657 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.292420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.304088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.304711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.304977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.305251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.305969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:48.306287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.308604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.323562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 60/126] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.324188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71088e-8000-465d-ad40-6a1df01b06f0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.329348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.344349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.378042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.378419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.378641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.378916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.382564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.383164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.384754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 67/127] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.385341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.487962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2e163-9181-494e-9804-39f13a748688 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.499292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeef057-30f4-4510-bdc0-1b057aaa6cd4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.524977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.525632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.525911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.526186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.527358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.529599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.531575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 61/128] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.532390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.551245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.552556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.553059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.554013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.555192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:48.555718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.557917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.574204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 68/129] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.575444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.603502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.641554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.661549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635661e7-d369-4cc7-97b7-00f8c861a6e8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.703867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.704627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.704968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.705310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.706664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.708957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.711255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 62/130] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:48.717704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.729537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6223e50-70cb-4478-b467-3af06b8feef0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.785384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.785384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.785384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.785384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/0f81252e23c341beb35dc2c378ed375c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.787902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, user_id=0f81252e23c341beb35dc2c378ed375c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.789430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.807567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 69/131] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:43:48 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/0f81252e23c341beb35dc2c378ed375c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:43:48.808215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.861835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ebc6-a89a-422e-b709-bf2b4c8bf8c0 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.900002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.911036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.911918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.912266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.912612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/a418650541634431838ac8d10049c192/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.914870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, user_id=a418650541634431838ac8d10049c192, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.916501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.937367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 63/132] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:43:48 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/a418650541634431838ac8d10049c192/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:43:48.938556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.050096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05871d-05ee-4922-bae8-b12af44c3d32 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.053091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:49.096772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.097747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.098291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.099729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.101331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.103244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880e44d-37c8-451f-98bd-1095a46e7940 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.105079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 70/133] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:49.107171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0329497-f006-44b3-81f4-01d9d3b18ba1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.107608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0329497-f006-44b3-81f4-01d9d3b18ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.108078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0329497-f006-44b3-81f4-01d9d3b18ba1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.109657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 71/134] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:49.110287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8bfd4-0b1a-4abf-b60e-ca5a2f9db961 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.111087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0329497-f006-44b3-81f4-01d9d3b18ba1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.156948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.157774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.158128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.158540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.159961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.161926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706283b-8000-4ee3-ae66-02dd97be76b6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.163925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 64/135] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:49.165270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.168430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.168430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.193515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:49.194958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:49.258866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0329497-f006-44b3-81f4-01d9d3b18ba1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.262453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:49.290966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 65/136] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:49.291934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.310834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.312665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.313094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.313506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] PATH_INFO: `/v3/projects/1cc9adf0ab9c4142bd223552de529932` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.315566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] RBAC: Authorizing `identity:get_project(project_id=1cc9adf0ab9c4142bd223552de529932)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.318730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.321852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 72/137] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:49 2022] GET /identity/v3/projects/1cc9adf0ab9c4142bd223552de529932 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:49.328635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.406446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:49.512052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a7bc9-e587-4973-a75a-877d02e2e9bd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.564326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.565192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.565620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.565899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] PATH_INFO: `/v3/projects/1cc9adf0ab9c4142bd223552de529932/users/a418650541634431838ac8d10049c192/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.568289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1cc9adf0ab9c4142bd223552de529932, user_id=a418650541634431838ac8d10049c192, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.570025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.587730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 73/138] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:43:49 2022] PUT /identity/v3/projects/1cc9adf0ab9c4142bd223552de529932/users/a418650541634431838ac8d10049c192/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:43:49.588484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf11aa-e063-41bb-9642-c63ba182ca71 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.668390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.669113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.669411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.669678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/a418650541634431838ac8d10049c192/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.676200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, user_id=a418650541634431838ac8d10049c192, role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.677787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.699169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 66/139] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:43:49 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/a418650541634431838ac8d10049c192/roles/f12fcc11a666405fa05a294664d03492 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:43:49.738348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.851795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:49.855889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53bd7-5aaf-476a-b3ee-e7efc67cc9b9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.905745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.906603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.906945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.907284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.908686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.910505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0775d4-b808-48ce-b401-fdd091f914eb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.912641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 74/140] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:49.918282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:49.976912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:50.057545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943124f-89ca-4a83-90e4-20829428561c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:50.101263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:50.102190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.102558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.102933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.103825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:50.105505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cf28d-99b8-4878-9be7-ffa226c33211 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:50.145323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 67/141] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:43:49 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:51.320541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d469a6-a662-462e-888a-93cd436f460c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.321442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d469a6-a662-462e-888a-93cd436f460c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.321871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d469a6-a662-462e-888a-93cd436f460c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.323551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 75/142] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:51.329954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98c9b-4827-4106-8b2d-3bafb94f4d2a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.330802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98c9b-4827-4106-8b2d-3bafb94f4d2a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.331221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e98c9b-4827-4106-8b2d-3bafb94f4d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.354892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d7e98c9b-4827-4106-8b2d-3bafb94f4d2a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:51.356366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d7e98c9b-4827-4106-8b2d-3bafb94f4d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.441704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 68/143] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:51.768396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd43b2-f1b4-4a6a-98b8-8c88d8cb5afe None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.769072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd43b2-f1b4-4a6a-98b8-8c88d8cb5afe None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.769488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd43b2-f1b4-4a6a-98b8-8c88d8cb5afe None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.771077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 76/144] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:51.788121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.788786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.789415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.814127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:51.815686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.850402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:51.907837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 69/145] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:52.051108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd43b2-f1b4-4a6a-98b8-8c88d8cb5afe None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.172338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd43b2-f1b4-4a6a-98b8-8c88d8cb5afe None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:52.216026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:52.216695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.216971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.217238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.231119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:52.236217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:52.237124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:52.237848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 77/146] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:52.244923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.391639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97670ec3-6d36-481c-9cbe-5d2ca5d1aa38 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:52.427535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.440693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:52.441337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.441612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.441881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.442587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:52.442890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:52.444309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:52.454041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 70/147] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:52.463459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.501088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:52.508748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:52.648193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd399bfa-06d4-4726-bab0-784f1ed542df None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:52.680720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f88d-a183-453b-a4fb-b2a9e5b8a241 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:52.712160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:52.712820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.713096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.713363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] PATH_INFO: `/v3/users/44804d7f61b9408bbd25cafbd9aab87d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.715088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] RBAC: Authorizing `identity:get_user(user_id=44804d7f61b9408bbd25cafbd9aab87d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:52.716552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:52.718239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 71/148] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/users/44804d7f61b9408bbd25cafbd9aab87d => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:52.723898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.795124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:52.795892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.796175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.796446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.823409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ddbc14b0-e863-4b4d-9ed9-ee8a4f55f77e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:52.828517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:52.829374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:52.830103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 78/149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:43:52.831559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af432-4791-40cf-9ff2-1674cf6edc9c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.831559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af432-4791-40cf-9ff2-1674cf6edc9c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.831559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af432-4791-40cf-9ff2-1674cf6edc9c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.832919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 79/150] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:52.839730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.840009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.840280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.883713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:52.885715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:52.911127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:52.953697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54873d2-2159-4a50-8a08-669b22e74f33 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.018832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 80/151] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:53.020149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:53.040185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.041282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.041757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.042251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.044851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.047066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.052826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 72/152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:53.063093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:53.075311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:53.210418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c134d-5379-4d1b-a98c-7ca80194bebb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.235689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0fe6bb-95e6-4029-9681-566064441025 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.285013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.285662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.285945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.286216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.286924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:53.287226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.288661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.292443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.293136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.293413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.293689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/44804d7f61b9408bbd25cafbd9aab87d/roles/3e412976f84c4f64914a78835f1b3566` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.300002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, user_id=44804d7f61b9408bbd25cafbd9aab87d, role_id=3e412976f84c4f64914a78835f1b3566)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.301497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.307315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 81/153] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:53.313157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:53.321366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 73/154] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:43:53 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/users/44804d7f61b9408bbd25cafbd9aab87d/roles/3e412976f84c4f64914a78835f1b3566 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:43:53.481127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:53.509304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6d051-83b2-47b1-8736-752d0f0842e2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.560913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.561634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.561913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.562181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] PATH_INFO: `/v3/users/0f81252e23c341beb35dc2c378ed375c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.563427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] RBAC: Authorizing `identity:get_user(user_id=0f81252e23c341beb35dc2c378ed375c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.564958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9988b3-2aba-4678-9168-a70b3069d00c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.566919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 82/155] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:53 2022] GET /identity/v3/users/0f81252e23c341beb35dc2c378ed375c => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:53.579584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:53.580093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e456d8-c0ef-4dea-bb30-6470e762eb86 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.580402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e456d8-c0ef-4dea-bb30-6470e762eb86 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.580589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e456d8-c0ef-4dea-bb30-6470e762eb86 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.581969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 83/156] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:53.587901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab5651-ef47-4fb7-9079-1c9d31c816dc None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.588179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab5651-ef47-4fb7-9079-1c9d31c816dc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.588459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab5651-ef47-4fb7-9079-1c9d31c816dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.615603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-afab5651-ef47-4fb7-9079-1c9d31c816dc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:53.616874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-afab5651-ef47-4fb7-9079-1c9d31c816dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.642779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:53.776980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 84/157] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:43:53.812910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454e4d-176d-44b1-ab7b-45f31393edfd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.857010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.857726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.858023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.858303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.891960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7862f3-c177-4cb6-9837-cde5e74425de None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.892266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7862f3-c177-4cb6-9837-cde5e74425de None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.892513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7862f3-c177-4cb6-9837-cde5e74425de None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.894022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 85/158] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:53.908331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.908830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.908830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.938000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.939975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.948589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:53.949821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.962208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.065139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 86/159] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:54.065948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.082797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 74/160] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:54.083998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.088993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.237574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cff24-6d20-49c7-8ce3-044c966cf52a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.265258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.276544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac30597-3448-4a24-8de2-7e8a6a21fe18 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.296080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.296750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.297016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.297287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.303178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:54.304672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:54.306606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 75/161] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:43:54.312660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.339183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.339767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.340044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.340406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.363302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:54.368595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:54.369426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:54.370169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 87/162] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:54.378651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.501061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.560441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4edd7-8d87-47a9-aab7-a079281f53c3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.610507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef228d-085b-4df6-9bb3-bc188db38d60 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.638878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.639583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.639862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.640133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.640844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:54.641149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:54.642694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:54.666275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 88/163] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:54.669069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.724508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.725782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.726283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.726817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.728164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=0f81252e23c341beb35dc2c378ed375c) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:54.728534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:54.730038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:54.760543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 76/164] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/role_assignments?user.id=0f81252e23c341beb35dc2c378ed375c&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:54.761514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:54.870793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.916096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9409d-0c76-4adf-9846-52de0f0b5a6d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.976546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.980182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce09a8-12a7-4300-b5d1-40ad7a9a9e06 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.992803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.993882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.994236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.994493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.015603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:55.024492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:55.025424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:43:55.026183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 89/165] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:55.027083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.054619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.055310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.055638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.055903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.077085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-96cd90b5-5db6-4e82-a430-a802a3a9d0e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:55.083835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:55.084714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:55.085489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 77/166] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:43:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:55.086651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cd4c-adb7-447d-8740-e4bb320127ea None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.086875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cd4c-adb7-447d-8740-e4bb320127ea None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.087144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cd4c-adb7-447d-8740-e4bb320127ea None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.096569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 78/167] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:55.097225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901cd4c-adb7-447d-8740-e4bb320127ea None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.224497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f573-508b-439c-a28f-430c6ad07c4a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.233207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901cd4c-adb7-447d-8740-e4bb320127ea None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.252827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.277222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.277929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.278210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.278604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.279284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:55.279645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.281144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.292378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 79/168] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:55.293379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.307118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.307792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.308074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.308350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.309719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.311297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa04f2b-04e4-4491-b3e8-cf87aee34f98 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.312943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 90/169] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:55.314166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.314477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.314751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.347479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:55.348697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:55.407094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25dfb1-c3f3-4fb8-8362-d9a0261a49d1 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.446772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.447407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.447687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.447963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.448640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:55.449009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.451200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.451680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.461296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 80/170] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:55.462176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.475669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 91/171] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:55.476484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.502650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.590523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d256f-cdf4-4f66-815f-bf88e529c4fa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.629557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.630228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.631546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.631546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.632343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.633943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.636009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 81/172] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:55.636843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.666669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282cf31-4da0-4517-a328-2a6e3bc15c0b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.742800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.744160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.744620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.745079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.745931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294d8e-7cc1-4e53-a51d-c7274817a1ed None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.747686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.788193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.789192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.789713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.790197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] PATH_INFO: `/v3/users/a418650541634431838ac8d10049c192` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.791822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] RBAC: Authorizing `identity:get_user(user_id=a418650541634431838ac8d10049c192)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.793474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.795515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 82/173] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/users/a418650541634431838ac8d10049c192 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:55.796699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.875476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.900446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.901897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.907925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a71f9-7b93-4548-a44e-f1e94db2d51d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.945057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.946357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.946939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.947350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.000869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.017914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 92/174] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:56.019203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.037628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.039603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5127ac-c8a3-4731-9911-86a73056433e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.144454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.161253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f1ae1-15ba-4cd8-bbbb-82d4c8f14b03 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.167792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 83/175] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:56.173786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f448e-37a4-4800-bc94-e955da5505e9 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.174167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f448e-37a4-4800-bc94-e955da5505e9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.174629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f448e-37a4-4800-bc94-e955da5505e9 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.176268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 84/176] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:56.177098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f448e-37a4-4800-bc94-e955da5505e9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.201628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.202651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.203070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.203478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.250560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.277623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.279085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.283989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f448e-37a4-4800-bc94-e955da5505e9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.320500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.321266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.321421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.321686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.334019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:56.338968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:56.339820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:56.341116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 85/177] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:56.345803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.373534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.389269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 93/178] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:56.390069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.465412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3e18-f19a-4b44-ba57-12a20172d117 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.495396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.502972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.503874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.504190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.504840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.506339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.508869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88098c44-ee0a-447b-80ac-66b882c24055 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.510989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 86/179] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.512393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30844078-59e5-4f06-8687-1f528de9137e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.512729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30844078-59e5-4f06-8687-1f528de9137e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.513089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30844078-59e5-4f06-8687-1f528de9137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.518667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b94f-865b-424e-ab36-69ce39c4862d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.531624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-30844078-59e5-4f06-8687-1f528de9137e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:56.533081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-30844078-59e5-4f06-8687-1f528de9137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:56.566535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.567193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.567470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.567738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] PATH_INFO: `/v3/projects/1cc9adf0ab9c4142bd223552de529932` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.569278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] RBAC: Authorizing `identity:get_project(project_id=1cc9adf0ab9c4142bd223552de529932)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.570800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804adc0-4df9-4067-8cef-1939c97eff1e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.572760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 94/180] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/projects/1cc9adf0ab9c4142bd223552de529932 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.573632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8874de-1cf6-4b85-aad8-115be2bd5b55 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.573913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8874de-1cf6-4b85-aad8-115be2bd5b55 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.574183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8874de-1cf6-4b85-aad8-115be2bd5b55 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.575423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 95/181] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:56.576227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8874de-1cf6-4b85-aad8-115be2bd5b55 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.617961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 87/182] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:56.619826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30844078-59e5-4f06-8687-1f528de9137e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.647288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.699009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8874de-1cf6-4b85-aad8-115be2bd5b55 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.730799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30844078-59e5-4f06-8687-1f528de9137e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.741237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.741863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.742144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.742441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.743118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:56.743787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.745170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.753715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 96/183] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.753715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.762263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.772480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.773438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.773691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.773973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.775516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.777318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.779023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 88/184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.782690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.874746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835109f-c645-49a9-b501-941f7b8da37f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.893342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.897077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce361a3-1ec8-4add-9539-a836a81a616c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:56.913945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.914611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.914894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.915165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.915906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=a418650541634431838ac8d10049c192) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:56.916216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.917740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7e723-f724-4c86-881e-8950b496a5ca None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.924211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 97/185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.925445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.925734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.926071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.937928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:56.938891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.938891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.939269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.939921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=1cc9adf0ab9c4142bd223552de529932, user.id=a418650541634431838ac8d10049c192) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:56.940219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:56.941865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec38d-7e9e-467d-bb6e-fe47e72581fa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:56.946423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:56.947685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:56.950002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 89/186] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=1cc9adf0ab9c4142bd223552de529932&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:56.951646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b20bf-b32a-4edf-b5dd-86a08d9e88c3 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.952126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b20bf-b32a-4edf-b5dd-86a08d9e88c3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.952647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b20bf-b32a-4edf-b5dd-86a08d9e88c3 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.954326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 90/187] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:43:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:56.955387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b20bf-b32a-4edf-b5dd-86a08d9e88c3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.050872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.078303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 98/188] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:57.079711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.116034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b20bf-b32a-4edf-b5dd-86a08d9e88c3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.167279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.168763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.169398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.169695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.187377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.190290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.192167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 91/189] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:57.192986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.239757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0139b-57de-4f92-ad35-cff86e5e381e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.258352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.283729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.284478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.284755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.285024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.285776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=a418650541634431838ac8d10049c192) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:57.286083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.287622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc4d14-c0d0-455a-bfa0-323c4d8b3c8a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.294254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 99/190] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/role_assignments?user.id=a418650541634431838ac8d10049c192&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=f12fcc11a666405fa05a294664d03492 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:57.297885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.298232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.298647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.320232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:57.321397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:57.349209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12c59a-1cca-41c0-b708-84f44af5ecf3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.371597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.391423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.391797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.391959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.392225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.404558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:57.410092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:57.411088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:57.411855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 92/191] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:57.412483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.420387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 100/192] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:43:57.421714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.560254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337ea2f-d45f-43a0-af04-bf634c76a9bf None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.569195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63085d35-4806-408c-aa3c-c10dc62cbd6a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.576547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.612540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.613128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.613406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.613673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.614361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:57.615799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.617282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.624066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.624704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.624984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.625257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.633312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 101/193] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:57.634851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.684261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.714257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.716137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.787398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475a210-5ab7-4f65-bccb-fbf0a4d8fe3b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.822391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.823341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.823567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.823865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.825024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.826684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.828520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 102/194] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:57.829294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.831738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 93/195] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:57.833404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.872030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.923976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.943416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7477f-7bf3-4580-8293-e26bfba70d09 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.946662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6d8e7-6d0c-4c84-a8c3-1d1219164f5b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.980855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.981497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.981771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.982039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.982636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.983274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.983370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.983692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.983961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.984727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.986339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 103/196] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:57.987132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.996165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:43:58.001147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:43:58.002187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:43:58.003001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 94/197] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:43:58.003663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.089790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30105-eb14-4f63-b288-3b2faafa1ba1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.092023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.105368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacfd4c-501d-4505-a786-92b2c9852a6d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.124872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.125537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.125771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.126028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.126774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.128231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.140337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.140976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.141262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.141517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.142742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.144188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.145847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 95/198] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.146640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.155474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 104/199] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:43:57 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:43:58.155845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.156621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.266019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac3416-07e6-458b-8cea-82e22296cb98 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.268636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee51d9-870f-4e20-b9a0-a3219cb3f216 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.303435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.304100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.304381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.304653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.305353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:58.305653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.307146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.312926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.313575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.313855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.314127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.321880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 105/200] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.321880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.346623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.403815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.405122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.414321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.442618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca16733-4984-43ed-995d-2e1113b6fbf2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.488580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.489212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.489489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.489751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.490532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=87d60cfbb2254c17957f1c396a14705d) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:58.490837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.492376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.499850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 106/201] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/role_assignments?user.id=87d60cfbb2254c17957f1c396a14705d&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.500545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.544617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 96/202] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:43:58.550704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.648560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.669688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b25191-0f38-4b78-8a27-a4407081f8d9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.686287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ced068-c9d3-4d89-aac3-9cc5300efb90 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.699903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.721548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.722527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.722945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.723362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] PATH_INFO: `/v3/users/44804d7f61b9408bbd25cafbd9aab87d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.724918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] RBAC: Authorizing `identity:get_user(user_id=44804d7f61b9408bbd25cafbd9aab87d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.727592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.730287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 107/203] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/users/44804d7f61b9408bbd25cafbd9aab87d => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.734228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.734945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.735221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.735564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.736786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.737018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.738482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.740737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 97/204] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.746837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:58.866198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a980e-bc9a-4e37-bd47-89ce57091424 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.867019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b9d33-c8db-4f0a-84e2-9f7319a3c2ee None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:58.904232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.905408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.906014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.906579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.907260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:58.907950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.908293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.908628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.909500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=87d60cfbb2254c17957f1c396a14705d) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:58.909885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.912860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:58.920186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 98/205] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/role_assignments?user.id=87d60cfbb2254c17957f1c396a14705d&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=f12fcc11a666405fa05a294664d03492 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:58.943615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:58.997129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:58.999216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:59.114632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 108/206] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:43:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:43:59.121166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:59.184189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:59.250328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acf355-d8c8-4198-97b0-a340fa2aedd3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:59.284990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:59.285643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:59.285919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:59.286187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.287363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:59.288829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f993dd1-8b2f-4908-b1d7-a3a954876311 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:59.290902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 99/207] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:43:59 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:43:59.296771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:59.388902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:59.448599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096cbff-1f3b-497b-ae44-f4349e50c9f0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:59.497864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:59.498869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:59.499313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:59.499726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.500833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60, user.id=44804d7f61b9408bbd25cafbd9aab87d) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:43:59.501280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:59.502919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d24e96-28d1-46c7-a2d5-85f03ec15dd2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:59.509986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 109/208] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:43:59 2022] GET /identity/v3/role_assignments?user.id=44804d7f61b9408bbd25cafbd9aab87d&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=3e412976f84c4f64914a78835f1b3566 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:00.861453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada167dd-3036-4d1e-b000-d32a926003ef None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:00.861960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada167dd-3036-4d1e-b000-d32a926003ef None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:00.861960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada167dd-3036-4d1e-b000-d32a926003ef None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:00.863357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 100/209] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:00.869642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d5d5-5ee4-445b-a38d-312dc446d1d7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:00.870146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d5d5-5ee4-445b-a38d-312dc446d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:00.870387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811d5d5-5ee4-445b-a38d-312dc446d1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:00.890808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5811d5d5-5ee4-445b-a38d-312dc446d1d7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:00.892028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5811d5d5-5ee4-445b-a38d-312dc446d1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:00.973107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 110/210] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:01.001058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3956-4db7-489c-80a4-32178e8fda2e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.001373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3956-4db7-489c-80a4-32178e8fda2e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.001606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c3956-4db7-489c-80a4-32178e8fda2e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.003419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 101/211] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:01.009498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.009865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.010266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.030661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:01.031871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.059474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03a302-d37b-434e-ae0a-e995d9c04700 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.059794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03a302-d37b-434e-ae0a-e995d9c04700 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.060019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03a302-d37b-434e-ae0a-e995d9c04700 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.061705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 102/212] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:01.067041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.067335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.067612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.086410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:01.087633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.090751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.109893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 111/213] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:01.167960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 103/214] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:01.287963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.396991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.413830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118e289-36e5-4489-b09d-39c4f0faba7b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.440435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.452852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.453502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.453777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.454045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.464497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:01.469614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:01.470888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 112/215] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:01.471712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.513531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177312e-9efd-4565-a47a-6b52fac1b4d8 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.515406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.553374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.554025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.554302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.554613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.555989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:01.557544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c733f-db3a-450f-9be0-ce37162fd3d1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:01.559097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 104/216] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:01.560425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64921ee-2d0c-4be6-b958-3900bcd1fd63 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.560711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64921ee-2d0c-4be6-b958-3900bcd1fd63 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.560983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64921ee-2d0c-4be6-b958-3900bcd1fd63 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.562286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 105/217] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:01.563108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64921ee-2d0c-4be6-b958-3900bcd1fd63 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.591362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905940-67c4-46d2-83ff-d3928fc506a5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.631182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.631854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.632131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.632402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.643423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:01.648316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:01.649189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:01.649960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 113/218] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:01.650637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.680920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64921ee-2d0c-4be6-b958-3900bcd1fd63 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.684565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.724774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.725387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.725768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.726236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.727254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:01.727549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:01.728941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d911c-9b7b-43c0-9a11-ef8fd593f8ac None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:01.742739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 106/219] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:01.745108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.745419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.745748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.763256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.767415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:01.768483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca74c0-3952-451f-842c-725a08d53f58 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.768878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.810152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.810803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.811085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.811362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.812029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:01.813544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:01.838396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 114/220] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:01 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:01.839319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.849598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 107/221] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:01.850809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.976121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.985272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281be3d0-c146-498c-ad34-170c88a704ee None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.986304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dcb27-0be7-4832-8a2e-6168d0949e0d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.029313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.030034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.030312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.030609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] PATH_INFO: `/v3/users/312895ecbb3346238ee66b0a6c095525` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.041493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.042709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.043599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.044094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.045243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:02.045924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.046197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] RBAC: Authorizing `identity:get_user(user_id=312895ecbb3346238ee66b0a6c095525)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.047821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.048007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.051444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 108/222] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/users/312895ecbb3346238ee66b0a6c095525 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.057264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.059265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 115/223] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.065957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.091431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.187455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fb38a-7609-466e-aea0-e4934629ea32 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.206633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b47c-0c84-4c9d-a06b-a5a0f997329d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.238098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.238709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.238989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.239261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.248438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.249159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.249442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.249844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.251150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.252644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.254466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 116/224] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.255371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.269385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.328130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.330001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.361246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.379279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6d1cb-4b93-4d69-8bc6-c2d68107d816 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.415313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.416120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.416512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.416856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.429249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:02.435030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:02.435960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:02.436800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 117/225] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:02.437534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.450072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 109/226] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:02.451023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.523653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.563918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471356d0-d465-4d53-9b1c-1f29360eb04f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.575092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29575efe-ae55-4557-81de-5ee961e7ce6c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.612949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.613603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.613875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.614138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.615306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.616813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.617931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.623615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 118/227] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.623615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.623818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.623818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.624464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.624945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:02.625248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.626658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.636005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 110/228] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.636899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.645674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.751113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53ec6-a64e-4a8e-94f8-63d6fe71d9b3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.768064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb76591-8252-418c-8d21-ee454de793fc None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:02.787717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.788275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.788569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.788813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.790047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.790706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.792393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.799547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 119/229] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:02.799547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.821248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:02.822307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.823450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.823932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/87d60cfbb2254c17957f1c396a14705d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.827004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, user_id=87d60cfbb2254c17957f1c396a14705d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:02.828710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:02.858763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 111/230] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:02 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/87d60cfbb2254c17957f1c396a14705d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:02.858763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:02.934127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.977475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656189a-9e13-45f7-92a5-f8c885debf59 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.059844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.060475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.060701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.060974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.067259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:03.068805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:03.070945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 120/231] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:03.072016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928f9cd-e2ba-44b4-b580-ab58190264df None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.076362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:03.115476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.116592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.117068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.117564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.118986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=312895ecbb3346238ee66b0a6c095525) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:03.119796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:03.121491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:03.128806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 112/232] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:02 2022] GET /identity/v3/role_assignments?user.id=312895ecbb3346238ee66b0a6c095525&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:03.305904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9ee2a-f990-471f-b294-1b3a5d9c7560 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.375123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.375713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.375996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.376268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.377483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:03.379034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da4b07-036e-4ed6-87b8-a4524b3d520a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:03.381058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 121/233] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:03 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:03.387187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:03.447063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:03.522848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b6aa-ff9b-4002-ac10-a1c79d1bef3f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.567739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.568434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.568713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.568984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/87d60cfbb2254c17957f1c396a14705d/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.571217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, user_id=87d60cfbb2254c17957f1c396a14705d, role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:03.572692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c13f-5c87-44a0-9871-600b99a6571d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:03.589479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 113/234] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:03 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/87d60cfbb2254c17957f1c396a14705d/roles/f12fcc11a666405fa05a294664d03492 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:04.608830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c8f9-7ba1-4e4e-88d1-a0031c91dca0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.609345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c8f9-7ba1-4e4e-88d1-a0031c91dca0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.609345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c8f9-7ba1-4e4e-88d1-a0031c91dca0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.611076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 122/235] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:04.619808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a374beb-3e05-4ef4-aee9-f2bb51383072 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.620304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a374beb-3e05-4ef4-aee9-f2bb51383072 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.620376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a374beb-3e05-4ef4-aee9-f2bb51383072 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.621955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 114/236] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:04.627831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.627992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.628185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.628279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.628369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.628524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.647919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:04.649137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:04.652587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:04.653774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:04.679389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:04.754579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 115/237] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:04.784036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 123/238] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:05.073524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.115236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.122605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.219889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22799f5c-307b-420a-88f0-eb4523cd90d1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.289256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.289918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.290196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.290491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.312234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:05.317214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:05.318125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:05.318929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 116/239] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:05.324724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.341855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd65c6-c3cf-4eca-8f5b-fdf30b7c8afa None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.405898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.406594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.406877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.407149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.409431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:05.411020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:05.412661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 124/240] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:05.418230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.450821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.484518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.497178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d688a-1213-49e0-ba87-4daa3397a770 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.582410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.583389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.583870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.584486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.586341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:05.587220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:05.589637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:05.607074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 117/241] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:05.613344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.665039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623f495-16d0-49b9-bc2a-f181fab20674 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.716435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.717480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.717963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.718450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.719290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:05.720939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8a5fb-8eaa-4a47-9fb8-8dc162bc16c3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:05.751654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 125/242] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:05 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:44:05.753497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccede5-cd3a-417e-80ef-5a812b4a506f None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.753995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccede5-cd3a-417e-80ef-5a812b4a506f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.754565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccede5-cd3a-417e-80ef-5a812b4a506f None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.756418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 126/243] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:05.765944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0f9-dafe-4bee-a6ef-92006c6cd522 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.766560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0f9-dafe-4bee-a6ef-92006c6cd522 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.767053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fd0f9-dafe-4bee-a6ef-92006c6cd522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.778892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddb85a-b59f-4216-af14-2ef50b7bedf3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.808052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-532fd0f9-dafe-4bee-a6ef-92006c6cd522 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:05.809658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-532fd0f9-dafe-4bee-a6ef-92006c6cd522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:05.821394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.822164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.822548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.822889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] PATH_INFO: `/v3/users/8e42d9e8443243ce80472e8e2583a4f2` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.840794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] RBAC: Authorizing `identity:get_user(user_id=8e42d9e8443243ce80472e8e2583a4f2)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:05.842752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:05.844347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 118/244] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/users/8e42d9e8443243ce80472e8e2583a4f2 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:05.850631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.866064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.955236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.964438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 127/245] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:05.965691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eaff6-34cf-4e53-8885-2b1b72b38d9c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.965977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eaff6-34cf-4e53-8885-2b1b72b38d9c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.966245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381eaff6-34cf-4e53-8885-2b1b72b38d9c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.967937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 128/246] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:05.981753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.981998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517712cf-14a5-4ea9-b8cc-8bb43c133a83 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.982775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.983359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.011289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:06.013324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:06.026126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.026820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.027103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.027380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.115896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 129/247] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:06.123227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.125189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.138772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.229925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 119/248] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:06.235556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.290924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.342380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77c1e9-94d5-4ab9-b34a-fda641816148 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.385482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.395594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.395594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.395900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.396137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.397683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.399243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.401364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 130/249] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:06.407512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.422655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8333ea-3fb1-4eb3-b24d-93f2d39a7f37 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.440764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.477370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.478423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.478703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.478973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.491514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:06.497284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:06.498169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:06.500309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:06.502835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 120/250] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:06.503969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.556384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0010e03-ea3a-4e2c-9ee2-e5df2d8fa4de None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.606440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.607454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.608441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.608929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.610268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=8e42d9e8443243ce80472e8e2583a4f2) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:06.611053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.613560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.625774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 131/251] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/role_assignments?user.id=8e42d9e8443243ce80472e8e2583a4f2&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=3e412976f84c4f64914a78835f1b3566 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:06.626928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.637599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ad615-7dda-4b51-9caf-fc85edd18548 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.681015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.681015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.681015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.681015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.693314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-42a0a721-8c50-49bf-9ede-ab054e81fa49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:06.698975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:06.699920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:06.700676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 121/252] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:06.701952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbf1b-7394-4aa7-9569-9bcfda2a3514 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.701952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbf1b-7394-4aa7-9569-9bcfda2a3514 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.702126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbf1b-7394-4aa7-9569-9bcfda2a3514 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.708746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 122/253] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:06.709621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbf1b-7394-4aa7-9569-9bcfda2a3514 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.726710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.749723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.824494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7fcd5-7488-42bd-ab16-a012d65dc9d4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.841475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbf1b-7394-4aa7-9569-9bcfda2a3514 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.884798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.885781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.886327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.886774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.887667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:06.888123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.888508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.889376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.889722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.890057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.890605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ed450-45af-4824-a4fd-b21950ee8974 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.890905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:06.891282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.892906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.903118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 123/254] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:06.905324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.905766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.906206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.916894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 132/255] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:06.918188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.937831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:06.939290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:07.061791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 124/256] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:07.062495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.102044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.114837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b783-7606-4ab9-b55d-c2bdf5bced6f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.167279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.187274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:07.187947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.188229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.188506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] PATH_INFO: `/v3/users/312895ecbb3346238ee66b0a6c095525` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.189658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] RBAC: Authorizing `identity:get_user(user_id=312895ecbb3346238ee66b0a6c095525)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:07.199725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:07.201739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 133/257] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/users/312895ecbb3346238ee66b0a6c095525 => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:07.207282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.251185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be97c4a-ea32-4b14-9da6-0a6b18d51dac None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.324742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:07.325940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.326478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.327010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.330868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:07.332573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:07.334460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 125/258] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:07.349726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.405165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ce471-f2fb-4033-b004-39c699b5369f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.470844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:07.471507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.471789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.472063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.474347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:07.475926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:07.477832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 134/259] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:07.478632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.522040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.536633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa0746-0f66-4cbe-92b3-405f36c72409 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.597209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.604389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:07.605023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.605302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.605576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.672230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0783c-450c-4e2d-8493-bf2e823c979d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.724064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:07.725938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:07.755915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:07.756588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.756857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.757130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.777033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:07.785322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:07.786194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:07.786999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 135/260] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:07.787644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.889364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.906331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 126/261] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:07.908092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.976830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccade46-b3f1-484f-b045-4c65b1c4316c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.030292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.035269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc982e-98b5-4f20-817a-5fcd567faf3d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.054500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.055183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.055460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.055730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/312895ecbb3346238ee66b0a6c095525/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.058976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, user_id=312895ecbb3346238ee66b0a6c095525, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.060484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.083145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 136/262] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:07 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/312895ecbb3346238ee66b0a6c095525/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:08.083979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.106639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.107285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.107560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.107880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.108671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:08.109032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.110542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.124696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 127/263] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:08.143622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.279308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ebabc-f9f7-486e-8330-f97b8918b4c6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.299057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.331249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.331922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.332196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.332463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.335087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.336577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.342399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 137/264] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:08.347478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.411798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb567f-2ca4-483c-be4a-cc89ec4676da None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.475921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.485201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.485868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.486150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.486475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] PATH_INFO: `/v3/users/87d60cfbb2254c17957f1c396a14705d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.491467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] RBAC: Authorizing `identity:get_user(user_id=87d60cfbb2254c17957f1c396a14705d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.493512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.495503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 128/265] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/users/87d60cfbb2254c17957f1c396a14705d => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:08.501460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.600588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71678e10-9591-4087-873e-a674106b36d8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.642776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9941c-19e8-46c9-b83b-79dd2b50f456 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.665234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.682157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.684398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.685425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.686420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.686985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.687938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=87d60cfbb2254c17957f1c396a14705d) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:08.688743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.689499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.689740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.690183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.691925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.708222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 138/266] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/role_assignments?user.id=87d60cfbb2254c17957f1c396a14705d&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:08.779390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.781462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3ef7-bffc-4593-94a6-6bde5c9c686c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.901885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 129/267] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:08.908430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.990786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989422f-6545-4cd7-945a-fa08e8cf68ab None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.991120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989422f-6545-4cd7-945a-fa08e8cf68ab None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.991332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989422f-6545-4cd7-945a-fa08e8cf68ab None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.993391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 130/268] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:09.000848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.001155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.001382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.036137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:09.037694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.056383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.086926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473394-e8bc-4460-8640-f93de64c4548 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:09.122361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.160594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:09.160913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 131/269] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:09.161770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.162261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.162811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.166878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:09.168723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d353a33d-ab57-4b41-8e32-a7b95cac0099 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:09.170871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 139/270] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:09.177205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:09.221266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9031c2-c012-4a58-9752-d4742ee0248f None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.222163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9031c2-c012-4a58-9752-d4742ee0248f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.222603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9031c2-c012-4a58-9752-d4742ee0248f None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.224882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 140/271] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:09.231134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.231861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.232322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.254211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:09.256075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.337648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 141/272] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:09.370650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69678965-1f43-4240-ab48-5a88c4c498d9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:09.412084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:09.413092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.413576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.414061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.415046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=87d60cfbb2254c17957f1c396a14705d) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:09.415555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:09.417260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:09.424845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 132/273] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:09 2022] GET /identity/v3/role_assignments?user.id=87d60cfbb2254c17957f1c396a14705d&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=f12fcc11a666405fa05a294664d03492 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:09.629819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:09.740998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:09.764931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.803954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc0cc6-2feb-4d01-8dc9-a7a96d688429 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:09.815717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.850152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:09.850935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.851085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.851364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.863718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6d603749-6b6a-4014-a08b-9e6362037d65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:09.869105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:09.869986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:09.870843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 142/274] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:09.871948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e41ec-1457-45fa-bcb2-7204e5812941 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.871948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e41ec-1457-45fa-bcb2-7204e5812941 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.872190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e41ec-1457-45fa-bcb2-7204e5812941 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.874403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 143/275] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:09.878973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.879308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.879542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.899479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc4486-8693-47c1-a411-0f5da641be4f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:09.899703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:09.900757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.937552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:09.938260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.938562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.938767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.939955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:09.941356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:09.943178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 133/276] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:09.947914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:09.974728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 144/277] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:09.975383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.054145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:10.085581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:10.086744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2e22-347a-44c4-ad31-c9eee4b5593b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.089619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf073744-9b9a-4cae-958e-29cca771cc15 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.127365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.128246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.128767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.129274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.130104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.130302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.131034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.131400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.131755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.132521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:10.132923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.133710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.134504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.146316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 134/278] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:10.152925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.170653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 145/279] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:09 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:10.268758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece4c54-f713-41d7-81fb-42742bf8873d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.270212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.307807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.308435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.308715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.308988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] PATH_INFO: `/v3/users/b6fe95beaf874d75a45a9f39d9133660` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.325520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b6fe95beaf874d75a45a9f39d9133660)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.327845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.330100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 135/280] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/users/b6fe95beaf874d75a45a9f39d9133660 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:10.336002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.338893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:10.415334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ef0a-82f1-425f-8f5b-119287872dec None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.460710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.461524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9990b0-c266-4bf7-a804-974a108985f3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.462045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.462606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.463077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.478854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:10.484420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:10.485277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:10.486648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 146/281] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:10.494533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.512150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.512792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.513074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.513347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.515565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:10.516659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, user.id=b6fe95beaf874d75a45a9f39d9133660, scope.system=all) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:10.516838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.518290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.526833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 136/282] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/role_assignments?user.id=b6fe95beaf874d75a45a9f39d9133660&scope.system=all&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:10.648422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb7202-4e32-4778-b898-9ddbbc0eb492 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.694279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.694952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.695232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.695502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.696217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:10.696523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.698003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.708402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 147/283] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:10.714232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:10.799836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:10.837343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab362010-1488-497e-a931-67205b8db9eb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:10.873932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:10.874624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:10.874905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:10.875177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] PATH_INFO: `/v3/users/8e42d9e8443243ce80472e8e2583a4f2` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:10.876339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] RBAC: Authorizing `identity:get_user(user_id=8e42d9e8443243ce80472e8e2583a4f2)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:10.877798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:10.879753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 137/284] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/users/8e42d9e8443243ce80472e8e2583a4f2 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:10.885183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:11.005518 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:11.039152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73bcaf-8c31-41f5-88b7-500ad95503d5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:11.084007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:11.084654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.084930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.085202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.086413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:11.087950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a656c4-8506-42be-b4c2-869025a50af8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:11.089750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 148/285] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:10 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:11.095449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:11.175603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ddd3-9ba5-40ba-b616-21f3a1470757 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.175916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ddd3-9ba5-40ba-b616-21f3a1470757 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.176146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7ddd3-9ba5-40ba-b616-21f3a1470757 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.177679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 149/286] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:11.209068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.209719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.210192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.230636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e58a5-00da-44ea-a98e-117a1a8c7411 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:11.242748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:11.244656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:11.271200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6697-a324-4441-8695-880b73a52467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:11.272383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6697-a324-4441-8695-880b73a52467 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.273616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6697-a324-4441-8695-880b73a52467 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.274269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6697-a324-4441-8695-880b73a52467 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/8e42d9e8443243ce80472e8e2583a4f2/roles/3e412976f84c4f64914a78835f1b3566` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.278761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6697-a324-4441-8695-880b73a52467 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, user_id=8e42d9e8443243ce80472e8e2583a4f2, role_id=3e412976f84c4f64914a78835f1b3566)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:11.280729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6697-a324-4441-8695-880b73a52467 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:11.293966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:11.312655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 138/287] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:11 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/users/8e42d9e8443243ce80472e8e2583a4f2/roles/3e412976f84c4f64914a78835f1b3566 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:11.361350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 150/288] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:11.761113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6697-a324-4441-8695-880b73a52467 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:11.897229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6697-a324-4441-8695-880b73a52467 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:11.962029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:11.963114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.963593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.964085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.983700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:11.989151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:11.990036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:11.990944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:11.998921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 139/289] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:12.003388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:12.096883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:12.128539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d497-0c72-4410-acfe-2fbd5f90eb36 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:12.176899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:12.177531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.177812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.178085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.178791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:12.179095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:12.180481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:12.189438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 151/290] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:12.195428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:12.281701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:12.344087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57737e72-3eb3-4d30-9b1d-84bae8c3a09d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:12.380346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:12.380922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.381198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.381472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] PATH_INFO: `/v3/users/312895ecbb3346238ee66b0a6c095525` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.382696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] RBAC: Authorizing `identity:get_user(user_id=312895ecbb3346238ee66b0a6c095525)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:12.384185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:12.386131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 140/291] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:12 2022] GET /identity/v3/users/312895ecbb3346238ee66b0a6c095525 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:12.392108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:12.527828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf7bf8-a39e-429e-8643-47daa24a722d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:12.565480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:12.568534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.568534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.568534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.579270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:12.666561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:12.668064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:12.784540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 152/292] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:12.790349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:12.933244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1c63-3c7e-4fcd-972b-285273dac501 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:12.969797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:12.991291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:12.992034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.992306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.992571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.000384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:13.001878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e39be-5561-4cae-8460-f8c2a282ac35 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:13.004029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 141/293] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:12 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:13.013820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:13.090662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.151026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcced69-50da-48ef-9128-a8e795d21eeb None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.151887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcced69-50da-48ef-9128-a8e795d21eeb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.152568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcced69-50da-48ef-9128-a8e795d21eeb None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.159779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 142/294] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:13.161931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c02-75df-4f9e-b27c-f33f65c3ff46 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.162475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c02-75df-4f9e-b27c-f33f65c3ff46 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.162961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a6c02-75df-4f9e-b27c-f33f65c3ff46 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.165227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 143/295] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:13.167341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa750e5-4c41-47ab-9b44-b24e7c9a82d3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.167875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa750e5-4c41-47ab-9b44-b24e7c9a82d3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.168359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa750e5-4c41-47ab-9b44-b24e7c9a82d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.180970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e29ca-7074-41c2-bb9e-8ddeadb24925 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:13.196282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-efa750e5-4c41-47ab-9b44-b24e7c9a82d3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:13.197428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-efa750e5-4c41-47ab-9b44-b24e7c9a82d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.243954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:13.244906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.245184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.245452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.246807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=312895ecbb3346238ee66b0a6c095525) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:13.247123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:13.249182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8444bd-0219-4f0b-b59f-e512eb7b4a34 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:13.258072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 153/296] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:13 2022] GET /identity/v3/role_assignments?user.id=312895ecbb3346238ee66b0a6c095525&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:13.259496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af171-8028-467b-9253-2070f2b96c52 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.259777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af171-8028-467b-9253-2070f2b96c52 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.260046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af171-8028-467b-9253-2070f2b96c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.281188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e64af171-8028-467b-9253-2070f2b96c52 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:13.283633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e64af171-8028-467b-9253-2070f2b96c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.373643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 144/297] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:13.395432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 154/298] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:13.635788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209b15-3bb6-4276-b3e2-890db8ac4648 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.636438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209b15-3bb6-4276-b3e2-890db8ac4648 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.636900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209b15-3bb6-4276-b3e2-890db8ac4648 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.638475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 145/299] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:13.645930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.646446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.646919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.671985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:13.673534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.681370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34209b15-3bb6-4276-b3e2-890db8ac4648 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:13.711590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.778084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 155/300] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:13.784566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:13.823851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34209b15-3bb6-4276-b3e2-890db8ac4648 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:13.849838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.874961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:13.875850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.876133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.876407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.890989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:13.900083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:13.901045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:13.901798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:13.902415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 146/301] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:13.907887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:13.911937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c97bf-6f0a-4248-94f7-8c90490e0687 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:13.963698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:13.964356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.965164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.965448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.966646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:13.969254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:13.971744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 156/302] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:13.977821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.013974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.041108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc041702-d2bd-4ee3-9f7a-9cb2444fe7b5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.081844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.082569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.082804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.083073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.083759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:14.084061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.085711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.097024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 147/303] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:14.097024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.108639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50409c-31f4-418d-82ab-bcb2490f55d7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.149079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.150057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.150338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.150634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] PATH_INFO: `/v3/groups` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.151124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.151564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.153165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.182422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 157/304] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Sep 13 19:44:13 2022] POST /identity/v3/groups => generated 229 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:44:14.182422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.231088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ed42e-4707-45a8-9c19-8df9b1b339a4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.270108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.270977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.271254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.271522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.284169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ea832107-86c6-49f5-97bb-26da3c7b44c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:14.289697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:14.290771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:14.291551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 148/305] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:14.292200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8410-34b4-4e06-b30b-9fbd4008b5a5 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.292962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8410-34b4-4e06-b30b-9fbd4008b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.292962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8410-34b4-4e06-b30b-9fbd4008b5a5 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.293539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.299168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 149/306] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:14.299168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd8410-34b4-4e06-b30b-9fbd4008b5a5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.310539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fd04-3bfd-47e6-98ca-54b7765dd2a4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.352118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.352692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.352968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.353237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] PATH_INFO: `/v3/users/bd89c8c545f14d44b9661c3532dd941b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.371742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] RBAC: Authorizing `identity:get_user(user_id=bd89c8c545f14d44b9661c3532dd941b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.373262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce310fa0-ef9c-467c-9431-3b458fe7de8d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.379828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 158/307] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/users/bd89c8c545f14d44b9661c3532dd941b => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:14.379828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.379828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.379828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.397344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:14.398638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:14.426565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd8410-34b4-4e06-b30b-9fbd4008b5a5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.428447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.465261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.465890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.466166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.466468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.467181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:14.467486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.468877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.478402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 159/308] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:14.478402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.492468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 150/309] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:14.493871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.588879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.614729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26f812-a97a-4d71-8b41-79db2c515377 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.636406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b7d26-e9fd-41f3-85ad-db6c9363b3e7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.653428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.654663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.655085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.655491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.656629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, user.id=bd89c8c545f14d44b9661c3532dd941b, scope.system=all) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:14.657540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.659688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.666790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 160/310] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/role_assignments?user.id=bd89c8c545f14d44b9661c3532dd941b&scope.system=all&role.id=3e412976f84c4f64914a78835f1b3566 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:14.682400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.683280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.683704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.684125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] PATH_INFO: `/v3/users/b6fe95beaf874d75a45a9f39d9133660` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.686022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] RBAC: Authorizing `identity:get_user(user_id=b6fe95beaf874d75a45a9f39d9133660)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.687944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.690470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 151/311] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/users/b6fe95beaf874d75a45a9f39d9133660 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:14.695986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.751815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.794199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:14.822347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fca61-274e-4a5b-a9d5-5763aec42f8a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.864040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.864724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.865080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.865325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] PATH_INFO: `/v3/system/users/b6fe95beaf874d75a45a9f39d9133660/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.866873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b6fe95beaf874d75a45a9f39d9133660, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:14.868658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:14.882168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 161/312] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Sep 13 19:44:14 2022] PUT /identity/v3/system/users/b6fe95beaf874d75a45a9f39d9133660/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:14.882435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45c33-70f9-4039-ba88-8a801278056d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:14.930677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:14.931400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.931675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.931989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.946580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:14.952519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:14.953482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:14.954244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 152/313] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:14.961510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:14.991729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.087075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6f3e0-f8a3-4975-8018-f6b630c6edb8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:15.138469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:15.139269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.139618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.139949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.140732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:15.141099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:15.142584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:15.152468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 162/314] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:15.158682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.264093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.275782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b768916-af66-43a7-9be8-909ddf3aad00 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:15.314422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:15.315417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.315909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.316363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] PATH_INFO: `/v3/users/8e42d9e8443243ce80472e8e2583a4f2` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.317674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=8e42d9e8443243ce80472e8e2583a4f2)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:15.319397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:15.321196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 153/315] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/users/8e42d9e8443243ce80472e8e2583a4f2 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:15.327717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.406595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.446574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85b39-1cc6-44bf-abbe-318300c53737 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:15.482900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:15.483533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.483809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.484080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.559409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:15.560998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:15.633346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.671553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 163/316] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:15.676796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.784628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74253e0c-b271-46db-b992-e0a5bb753cd4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:15.823160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:15.824216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.824707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.825158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.826568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:15.828225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482feeac-c01a-40ac-8361-3d7a2c8a7c71 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:15.830393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 154/317] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:15.839493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.971555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bcd33-9455-4e99-836e-7ef915004055 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:16.025573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:16.026205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.026512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.026787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.027527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a, user.id=8e42d9e8443243ce80472e8e2583a4f2) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:16.027833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:16.029305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b6b6-0036-4b7b-a517-d8f2d6d09657 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:16.036103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 164/318] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/role_assignments?user.id=8e42d9e8443243ce80472e8e2583a4f2&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=3e412976f84c4f64914a78835f1b3566 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:16.491082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1974-3f5f-4f8b-9d2b-7f415f99f357 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.491616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1974-3f5f-4f8b-9d2b-7f415f99f357 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.491893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c1974-3f5f-4f8b-9d2b-7f415f99f357 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.493551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 155/319] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:16.499818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edc38-ca36-4357-8d43-44dd6532351e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.500524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edc38-ca36-4357-8d43-44dd6532351e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.500524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edc38-ca36-4357-8d43-44dd6532351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.520692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-112edc38-ca36-4357-8d43-44dd6532351e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:16.521926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-112edc38-ca36-4357-8d43-44dd6532351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:16.560239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:16.612830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 165/320] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:16.887942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c1974-3f5f-4f8b-9d2b-7f415f99f357 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:16.919952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:16.920232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144465b-8c76-4acd-82e6-a78297222752 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.920322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144465b-8c76-4acd-82e6-a78297222752 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.920573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144465b-8c76-4acd-82e6-a78297222752 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.922080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 166/321] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:16.927182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.927348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.927617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.946523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:16.947718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:17.009141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c1974-3f5f-4f8b-9d2b-7f415f99f357 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.024727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 167/322] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:17.046795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.047473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.047743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.048008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.049309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.050900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8b90c-cf35-43ef-b822-2cb4abecba9f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.052874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 156/323] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:17.058528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.166336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38b6dc-f6c3-4e72-855a-30a4b777cc57 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.166609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38b6dc-f6c3-4e72-855a-30a4b777cc57 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.166833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38b6dc-f6c3-4e72-855a-30a4b777cc57 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.168559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 157/324] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:17.173671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.173958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.174581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.186135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.193035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:17.194257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:17.196542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f67d00-5995-4bc5-8606-7a0cfbd391bb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.233175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.233967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.235115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.235388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] PATH_INFO: `/v3/groups` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.236047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.237530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.257033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 168/325] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Sep 13 19:44:17 2022] POST /identity/v3/groups => generated 226 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:17.279794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 158/326] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:17.286321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.403314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281bf7-80f0-4799-89b8-9f6b400b964d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.441856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.442424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.442720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.442994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.458937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:17.467599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:17.468565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:17.469382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:17.470029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 169/327] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:17.478240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.513562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.547020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.564151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.608445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c23509-e28f-4d9a-b1b0-51b4004f0b83 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.648350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.648987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.649269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.649541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.650220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:17.650563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.651963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.666439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 159/328] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:17.668631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.688021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de43b-f99a-4fff-87b2-c1e4805584aa None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.731605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.732329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.732677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.733078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.746713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:17.752009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:17.752887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:17.753668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 170/329] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:17.764705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.794841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.811845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c907d82-efb9-400d-ac67-b78407186e42 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.860914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.872181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.873190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.873658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.874107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] PATH_INFO: `/v3/users/bd89c8c545f14d44b9661c3532dd941b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.875424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] RBAC: Authorizing `identity:get_user(user_id=bd89c8c545f14d44b9661c3532dd941b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.878835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.881799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 160/330] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/users/bd89c8c545f14d44b9661c3532dd941b => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:17.887368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:17.898894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e185b-66d6-4b05-87fe-c1a928cff4f5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.945115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.946486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.946961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.947417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.948285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:17.949276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.951173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.963652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 171/331] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:17.969453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:18.018311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35f9b-b747-4444-9136-809cd3971773 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:18.065806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:18.066666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.066897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.067177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] PATH_INFO: `/v3/system/users/bd89c8c545f14d44b9661c3532dd941b/roles/3e412976f84c4f64914a78835f1b3566` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.068773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bd89c8c545f14d44b9661c3532dd941b, role_id=3e412976f84c4f64914a78835f1b3566)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:18.070272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:18.083793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 161/332] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Sep 13 19:44:17 2022] PUT /identity/v3/system/users/bd89c8c545f14d44b9661c3532dd941b/roles/3e412976f84c4f64914a78835f1b3566 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:18.106686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2d2a5-08bd-4995-b028-2633795242c1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:18.146863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:18.147876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.148353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.148810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] PATH_INFO: `/v3/users/b6fe95beaf874d75a45a9f39d9133660` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.150234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] RBAC: Authorizing `identity:get_user(user_id=b6fe95beaf874d75a45a9f39d9133660)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:18.151969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c48ba9-e892-4f65-8521-53c474d8a566 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:18.154271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 172/333] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:17 2022] GET /identity/v3/users/b6fe95beaf874d75a45a9f39d9133660 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:18.161763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:18.197921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:18.308290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2d5b9-c54b-402c-bad3-f7d4359856c8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:18.350025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:18.350644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.350985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.351267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.352047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] RBAC: Adding query filter params (role.id=753c4dc548cd4aeabce829afb96fed52, user.id=b6fe95beaf874d75a45a9f39d9133660, scope.system=all) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:18.352358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:18.353969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad373e3f-4dca-4746-b73e-540bddd4a4a5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:18.369545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 162/334] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Sep 13 19:44:18 2022] GET /identity/v3/role_assignments?user.id=b6fe95beaf874d75a45a9f39d9133660&scope.system=all&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:20.117138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761332f-8a9d-4691-856b-31549bc51472 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.118041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761332f-8a9d-4691-856b-31549bc51472 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.118324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9761332f-8a9d-4691-856b-31549bc51472 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.120208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 173/335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:20.132592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0c9b-f412-43f9-ad32-94fa8ca806aa None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.133115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0c9b-f412-43f9-ad32-94fa8ca806aa None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.133355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d0c9b-f412-43f9-ad32-94fa8ca806aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.158000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1d3d0c9b-f412-43f9-ad32-94fa8ca806aa None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:20.159265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1d3d0c9b-f412-43f9-ad32-94fa8ca806aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:20.237422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 163/336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:20.425063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613eb2-b170-4845-aa49-97d5f8b6f5b4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.425722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613eb2-b170-4845-aa49-97d5f8b6f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.426214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37613eb2-b170-4845-aa49-97d5f8b6f5b4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.427875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 174/337] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:20.433939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.434229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.434530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.458012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:20.459241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:20.461360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:20.612493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 164/338] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:20.658105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37613eb2-b170-4845-aa49-97d5f8b6f5b4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:20.661797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:20.787016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37613eb2-b170-4845-aa49-97d5f8b6f5b4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:20.835146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:20.836212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.836715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.837205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.860139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:20.865566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:20.866542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:20.867313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:20.870098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 175/339] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:20.881039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.007691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525abc83-dd64-4c91-846a-90ad6b40086c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:21.052963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:21.053509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.053790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.054063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.054790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:21.055096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:21.056485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35d154-43c7-4638-a2f6-7751a6d30e9e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:21.067828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 165/340] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:21.068135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.078877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.157442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.200656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1f9f9-1c01-4491-b4b5-1ff4346f461d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:21.238585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:21.239213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.239491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.239754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.253903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78138c-afb6-4343-9e21-bc163baf6697 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.254857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78138c-afb6-4343-9e21-bc163baf6697 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.255430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78138c-afb6-4343-9e21-bc163baf6697 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.257381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 166/341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:21.266135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.266842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.267322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.295337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:21.296909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:21.328823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:21.330237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:21.406963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 167/342] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:21.408399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.425858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.481087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 176/343] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:21.489826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.548747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698aef7-144e-4bba-adf0-edc7bcb7d5a4 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:21.589857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.601277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:21.601588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.601828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.602094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.627877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-507bc5b5-5e4e-4646-9f66-d0f97259f51e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:21.633694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:21.634608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:21.635368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:21.642826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 168/344] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:21.645262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39a26-3207-499c-ac5b-d71a0fc82083 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.645541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39a26-3207-499c-ac5b-d71a0fc82083 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.645807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d39a26-3207-499c-ac5b-d71a0fc82083 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.648726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 169/345] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:21.653301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d39a26-3207-499c-ac5b-d71a0fc82083 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.711840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17cbcd-0dc0-44ca-a788-00714b3f45e8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:21.772480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:21.774548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.775215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.776851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.778946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:21.781302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a39e59-31e0-438e-accb-3e64c1a5813e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:21.785600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 177/346] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:21.787787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077779-8f01-4409-8734-b586cca94092 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.788981 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077779-8f01-4409-8734-b586cca94092 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.789637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab077779-8f01-4409-8734-b586cca94092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.824724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ab077779-8f01-4409-8734-b586cca94092 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:21.827050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ab077779-8f01-4409-8734-b586cca94092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:21.873936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d39a26-3207-499c-ac5b-d71a0fc82083 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:21.891331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.926209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:21.927214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.927651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.928063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.928868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:21.929287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:21.930925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:21.945339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 170/347] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:21.946522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:21.952867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 178/348] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:21.953967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab077779-8f01-4409-8734-b586cca94092 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.031886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.100181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab077779-8f01-4409-8734-b586cca94092 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.145936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d2b7-5a01-43b7-9b99-9b94027469b9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.147544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.148435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.148858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.149258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.178887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.224659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.226750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.272939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.274034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.274551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.275035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.295883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.298705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.302203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 171/349] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:22.303662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.357551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 179/350] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:22.366543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.440460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.493049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0c57-cd1d-4439-8f3b-73b72e71250c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.534119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1f62-d508-4b7d-b638-6d5a1bda28cd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.549519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140644-9b85-4585-ae76-5893015e580f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.550182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140644-9b85-4585-ae76-5893015e580f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.550881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140644-9b85-4585-ae76-5893015e580f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.551158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07140644-9b85-4585-ae76-5893015e580f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.557089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.567357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07140644-9b85-4585-ae76-5893015e580f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-07140644-9b85-4585-ae76-5893015e580f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:22.572562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:22.573438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:22.574234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 180/351] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:22.575106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140644-9b85-4585-ae76-5893015e580f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.596932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.598016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.598539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.599006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.615014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:22.621005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:22.621896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:22.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 172/352] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:22.623374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.699814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07140644-9b85-4585-ae76-5893015e580f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.729958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.752566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.753200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.753479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.753748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.754565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:22.754874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.756359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12bf69-76dd-4743-a0ee-10ecb9844b96 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.762574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 181/353] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:22.764108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f805-24a9-4e35-973f-05a85ab2e5e7 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.764108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f805-24a9-4e35-973f-05a85ab2e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.764366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05f805-24a9-4e35-973f-05a85ab2e5e7 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.767271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 182/354] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:22.767271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f805-24a9-4e35-973f-05a85ab2e5e7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.773374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c176b-6da0-4388-8bdd-927b991e9f25 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.825638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.826564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.826851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.827125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.828909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.829917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.832307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 173/355] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:22.833494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.898271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.926713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05f805-24a9-4e35-973f-05a85ab2e5e7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.970865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.971510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.971794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.972122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.972820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:22.973171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.974313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ddeb-ac56-4a2b-947c-873fef861273 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.974681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1b259-c93b-44db-be62-231dec19ad53 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.985291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 183/356] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:22.990120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.990120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.990120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.008576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:23.010504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:23.020825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.021456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.021735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.022018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.022824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:23.023130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.024548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.034797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 174/357] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:23.035681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.061248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.099421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 184/358] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:23.101116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.180503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72552b5c-9986-4098-a5c9-05fccd65623b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.188883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.223409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.223977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.224256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.224531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.225839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.227337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.228951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 175/359] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:23.229724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.245435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593720bc-c8be-440e-9b39-dba6824a81a1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.296677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.297394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.298296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.298642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] PATH_INFO: `/v3/users/bd89c8c545f14d44b9661c3532dd941b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.300166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] RBAC: Authorizing `identity:get_user(user_id=bd89c8c545f14d44b9661c3532dd941b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.302303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfa894-5fe1-4906-bfbf-57bb2c1a584d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.305262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 185/360] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/users/bd89c8c545f14d44b9661c3532dd941b => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:23.306770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd94e6-8aa7-44fe-ad3b-c8e88006c4da None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.307324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd94e6-8aa7-44fe-ad3b-c8e88006c4da None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.307604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd94e6-8aa7-44fe-ad3b-c8e88006c4da None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.309730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 186/361] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:23.310509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd94e6-8aa7-44fe-ad3b-c8e88006c4da None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.360935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd283eb-3bef-4108-ae02-1dc9472548eb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.394925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.419229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.420302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.420822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.421309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.479908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.484880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd94e6-8aa7-44fe-ad3b-c8e88006c4da None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.507588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.509060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5649a1-9401-4c8f-9f72-a5a7cd5abc27 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.547881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.548545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.548823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.549093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.549845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:23.550181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.551705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.558267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 187/362] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=f12fcc11a666405fa05a294664d03492 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:23.559001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.671005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 176/363] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:23.672833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.673348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.674021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.698739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:23.700563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:23.710512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483baa7-2d74-48da-9ee5-3979c8980c2c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.724659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.765172 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.766176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.767077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.767077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.767830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] RBAC: Adding query filter params (role.id=3e412976f84c4f64914a78835f1b3566, user.id=bd89c8c545f14d44b9661c3532dd941b, scope.system=all) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:23.768163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.769688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.784064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 188/364] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/role_assignments?user.id=bd89c8c545f14d44b9661c3532dd941b&scope.system=all&role.id=3e412976f84c4f64914a78835f1b3566 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:23.785050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.807318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 177/365] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:23.808567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.818590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.957359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c9c4b-f113-4299-aea6-0864e70197df None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.960090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539a72-950b-4476-b203-4de4c83738bd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.021266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.022294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.022608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.022881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.024124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.025775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.025907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.026452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.026735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.027004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.028466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 178/366] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:24.037292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.054600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:24.063585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:24.064534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:24.065330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 189/367] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:24.075092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.145686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.171548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.244890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ffb2b-0773-4e01-8eaa-df0c65853f35 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.280871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa52e-eaa3-45fe-82f1-1d7f939c2d81 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.327420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.329203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.329811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.330283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.332253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.334253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.335319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.335319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.335547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.336191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:24.336500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.336962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.337907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.340638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 179/368] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:24.349856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.363710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 190/369] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:24.364601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.489388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.507903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f852ce0-d5b5-412e-9271-e6f858e1b0b7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.509981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33281ba5-e461-4e97-bc69-dac858e4e6bb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.561477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.561663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.562097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.562197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.562405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.562511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.562664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.562807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.563513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:24.563827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.565475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.571898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.573239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 191/370] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:24.574155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.576059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:24.581307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:24.582223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:24.583044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:24.586570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 180/371] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:24.588820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.713049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786d9ad-6a77-4570-b5f3-44135305ed76 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.732219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e820642-5667-40eb-8a18-085ac3ba66c4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:24.766069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.768520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.768811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.769078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.791727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:24.792362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.792633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.792898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.793585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:24.793883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.795424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.814570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 181/372] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:24.819116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.819305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.900644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:24.902123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:24.931243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.038755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340a26f-f059-423b-884e-34220b889d15 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.082269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 192/373] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:25.088164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.106910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.107634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.107913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.108188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.175880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.198563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.200640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.255638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae15a4e-7147-4d04-93fe-291620b9e37d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.300760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.316378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.317046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.317312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.317589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.318797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.320260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.324653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 193/374] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:25.329961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.370040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 182/375] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:25.377401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.491482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a9291-c807-4f2d-90ed-69cbe541eab0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.516748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.535911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.536571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.536854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.537130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.538274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.539783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.541680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 194/376] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:25.541830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06991f03-d312-4ed2-96db-04ff56875fb7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.568528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.591992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.592652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.592929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.593198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] PATH_INFO: `/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.595032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=02e765af7d454a8f9fe205c67fe1fdbb)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.596503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.602740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 183/377] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:25.603981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.657175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.716511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db4932-9f0c-4a92-a3b8-29dd5689e0f0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.750419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb491-ecb0-417c-a46d-793dce4e2aa5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.756549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.757180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.757458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.757739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.758508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:25.758820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.760408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c137d87-77c8-4efe-9cb4-0850bea32a01 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.770095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 195/378] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=f12fcc11a666405fa05a294664d03492 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:25.771575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90f31-c781-4f8a-821a-226ce413f1f0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.771958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90f31-c781-4f8a-821a-226ce413f1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.772235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90f31-c781-4f8a-821a-226ce413f1f0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.773725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 196/379] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:25.779877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.780236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.780432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.803598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:25.804832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:25.806065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.806758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.807036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.807531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] PATH_INFO: `/v3/groups/2a3e77a28e4f4a1084a57c89360c6592` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.814887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] RBAC: Authorizing `identity:get_group(group_id=2a3e77a28e4f4a1084a57c89360c6592)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.816751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.819397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 184/380] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/groups/2a3e77a28e4f4a1084a57c89360c6592 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:25.827568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:25.895401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.945746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 197/381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:26.004343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:26.014445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fc5dc-a0a7-4908-84a6-9c6d02da4d10 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:26.056774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:26.057401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.057674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.057943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.058720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] RBAC: Adding query filter params (group.id=2a3e77a28e4f4a1084a57c89360c6592, role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=02e765af7d454a8f9fe205c67fe1fdbb) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:26.059025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:26.060546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c232113-acfa-4ab2-bdca-82bc5ffd871d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:26.067292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 185/382] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:25 2022] GET /identity/v3/role_assignments?group.id=2a3e77a28e4f4a1084a57c89360c6592&scope.project.id=02e765af7d454a8f9fe205c67fe1fdbb&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:26.319925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:26.456212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854bb17-ff7b-4c2a-853c-ae4dfeffe2c2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:26.480236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ed08-7405-4f9f-b823-a4cda58bef28 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.480885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ed08-7405-4f9f-b823-a4cda58bef28 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.481357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5ed08-7405-4f9f-b823-a4cda58bef28 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.484079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 186/383] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:26.490047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.490328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.490624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.499126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:26.500096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.500573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.501078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.514198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:26.514631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:26.515488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:26.520047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:26.520930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:26.521714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:26.524296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 198/384] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:26.530788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:26.594680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:26.633489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 187/385] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:26.691708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fb60b-d322-4eae-b889-ccc0306388d4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:26.746633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:26.747281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.747559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.747827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.748599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:26.748900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:26.750316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:26.762312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 199/386] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:26.768933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:26.884797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:26.898167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac2669-3c36-4973-8987-2e5a6a31066c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:26.943020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:26.943734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.944018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.944293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.945425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:26.946915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:26.948841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 188/387] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:26.956932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:26.999544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.141059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e365c7-d3e5-46cd-b001-e033fe5c3b4c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.153669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.178042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b54db-b96e-48eb-98bc-dc5bff16c25f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.178883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.179648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.180031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.180477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.182287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.184118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.186256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 200/388] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:27.192654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.260434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.261472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.261945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.262284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.282751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:27.287680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:27.288783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:27.289791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 189/389] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:27.296961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.339479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.403176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa96c3-2110-48e6-b3b8-8955829f6ce2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.484398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.485415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.485875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.486346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/groups/768771a0b3064733b184f4f6052255fc/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.489312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, group_id=768771a0b3064733b184f4f6052255fc, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.491961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a59429-63c1-4c2b-aa7b-88465cfd6172 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.511773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b440e-231a-4f39-b44e-1d51fb315c03 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.514835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 201/390] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Sep 13 19:44:27 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/groups/768771a0b3064733b184f4f6052255fc/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:27.517203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cfaef-1142-4fed-91f6-f9eb832dcc3c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.517929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cfaef-1142-4fed-91f6-f9eb832dcc3c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.518655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cfaef-1142-4fed-91f6-f9eb832dcc3c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.521162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 202/391] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:27.528781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.529680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.530313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.551864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:27.553711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:27.561000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.561737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.562018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.562290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.563067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:27.563360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.565356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.576737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 190/392] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:27.584366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.588753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.637245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 203/393] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:27.684210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.708513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4f9cf-78b8-489e-a0d7-6740021ed81c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.746425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.747195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.747475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.747783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.749562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.751217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.757168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 191/394] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:27 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:27.763677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.909212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec276825-2309-46ce-97a0-7cf9e80e6d29 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:27.946191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.947269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.947807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.948273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.949619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.951403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.953595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 204/395] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:27 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:27.959379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:27.980097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.034768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:28.121709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fed831-ea29-44e5-b421-6dbaeed1aee6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.182623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c32c7c-2246-49c1-a0ad-c75191882e88 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.204164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.205053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.205471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.205877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.231479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:28.239940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:28.240821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:28.242074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 205/396] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:28.250817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.276233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.276935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.277217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.277494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/groups/768771a0b3064733b184f4f6052255fc/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.281174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7cf1c0e7a9b40d48890395c76780e60, group_id=768771a0b3064733b184f4f6052255fc, role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.282669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:28.305933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 192/397] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Sep 13 19:44:27 2022] PUT /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60/groups/768771a0b3064733b184f4f6052255fc/roles/f12fcc11a666405fa05a294664d03492 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:28.389175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:28.401654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd228a-14f4-4835-aae1-9b3d2cd5d0c5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.446262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.446974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.447242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.447570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.448265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:28.448621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.450089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:28.461721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 206/398] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:28.467657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.477694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:28.617881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946c9fa-d129-4046-90d6-8a1f93bd2903 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.656634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.657298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.657578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.657851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.659078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.660574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:28.662777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 193/399] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:28 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:28.668526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.774499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f069881-bb87-4602-af5c-362ddbbf871f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.814295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.814987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.815269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.815543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.816687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.818210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4e0a6-9100-4f9c-b9d9-ae37cd5995de None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:28.819902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 207/400] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:28 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:28.825827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.915666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:28.942681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f07a-2b15-4e53-81e8-7c7e4f8ba591 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.984624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.985327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.985607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.985885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/groups/768771a0b3064733b184f4f6052255fc/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.987924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, group_id=768771a0b3064733b184f4f6052255fc, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.989394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b5fe3-9280-4795-bd74-baa7c759ae5b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:29.010563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 194/401] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Sep 13 19:44:28 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/groups/768771a0b3064733b184f4f6052255fc/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:29.016015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d3ce-21d2-49e0-a001-dc3a9974a8e1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.016294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d3ce-21d2-49e0-a001-dc3a9974a8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.016573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d3ce-21d2-49e0-a001-dc3a9974a8e1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.018128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 208/402] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:29.024963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bdee-5718-4dd0-b5e2-48d41bc67c77 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.025264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bdee-5718-4dd0-b5e2-48d41bc67c77 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.025447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2bdee-5718-4dd0-b5e2-48d41bc67c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.051328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fea2bdee-5718-4dd0-b5e2-48d41bc67c77 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:29.052931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fea2bdee-5718-4dd0-b5e2-48d41bc67c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:29.139371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 195/403] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:29.285353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dd243-efb2-42d6-81a6-7dcfc4aa5f97 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.285686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dd243-efb2-42d6-81a6-7dcfc4aa5f97 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.285854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dd243-efb2-42d6-81a6-7dcfc4aa5f97 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.287491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 209/404] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:29.292689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.293269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.293735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.323449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:29.325186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:29.405691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:29.414271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 196/405] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:29.541346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dd243-efb2-42d6-81a6-7dcfc4aa5f97 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:29.571148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:29.680049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dd243-efb2-42d6-81a6-7dcfc4aa5f97 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:29.721049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:29.721821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.722197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.722506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.743494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:29.745621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:29.750929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:29.752086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:29.753195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 210/406] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:29.758153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:29.902839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:29.913803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6072ac4-a068-441a-ac7e-b655e9fc96a8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:29.927914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8022fb-088d-4cc1-8fe5-50d2477d6304 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:29.966144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:29.966924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.967208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.967493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.968232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:29.968758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:29.970599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:29.981112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 211/407] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:29.987674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.019811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.020494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.020779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.021059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.045194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:30.059259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:30.060887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:30.062424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:30.070614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 197/408] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:30.079487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.150871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946b457-7d0b-42ef-bf77-1cd851e1aa56 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.176961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.218549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1ff4f-d530-4a2c-99fd-a8f4001b807e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.256902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.257582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.257945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.258227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.258941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:30.259246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.260640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.269949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 198/409] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:30.271104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.272203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.272668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.273163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.275607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.283711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.286242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.289729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 212/410] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:29 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:30.295462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.337222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.379913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a0990-9593-4e9f-a137-bd146e94726f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.404186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.416802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.417741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.418196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.418679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] PATH_INFO: `/v3/groups/2a3e77a28e4f4a1084a57c89360c6592` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.420069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] RBAC: Authorizing `identity:get_group(group_id=2a3e77a28e4f4a1084a57c89360c6592)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.421774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.423956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 199/411] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:30 2022] GET /identity/v3/groups/2a3e77a28e4f4a1084a57c89360c6592 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:30.434219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.611915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8f323-89c0-41ce-bb07-42276b0468d3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.625329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301428-ff71-40ed-8d79-152212b14cb1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.681377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.682458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.683174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.684284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] PATH_INFO: `/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.686421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] RBAC: Authorizing `identity:get_project(project_id=02e765af7d454a8f9fe205c67fe1fdbb)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.688654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6e9f-ede4-43bf-8fc4-d588917f15f1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.690232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.690959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.691240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.691511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.692276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 200/412] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:30 2022] GET /identity/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:30.693760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99c92-f58d-4eb4-b9a2-31738f8e29bd None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.694259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99c92-f58d-4eb4-b9a2-31738f8e29bd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.694762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf99c92-f58d-4eb4-b9a2-31738f8e29bd None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.700689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.700689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafff44-0529-46bf-bd0c-418119bb7ea2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.701179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 201/413] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:30.701903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 213/414] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:30 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:30.702584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf99c92-f58d-4eb4-b9a2-31738f8e29bd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.707659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.707659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.707659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.731232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:30.732625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:30.775337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.814403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 214/415] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:30.814403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.826241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.856621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf99c92-f58d-4eb4-b9a2-31738f8e29bd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.896240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.897307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.897802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.898476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] PATH_INFO: `/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb/groups/2a3e77a28e4f4a1084a57c89360c6592/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.900619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] RBAC: Authorizing `identity:create_grant(project_id=02e765af7d454a8f9fe205c67fe1fdbb, group_id=2a3e77a28e4f4a1084a57c89360c6592, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.902277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf58ca-629d-4136-b4b9-2fc1961d8a3e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.921620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 202/416] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Sep 13 19:44:30 2022] PUT /identity/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb/groups/2a3e77a28e4f4a1084a57c89360c6592/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:31.007212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b07138-7c46-4222-941b-0b2cf98ff0f8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:31.057623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:31.058355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.058673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.058947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/groups/768771a0b3064733b184f4f6052255fc/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.061535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a, group_id=768771a0b3064733b184f4f6052255fc, role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:31.063552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:31.080896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 215/417] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Sep 13 19:44:30 2022] PUT /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a/groups/768771a0b3064733b184f4f6052255fc/roles/f12fcc11a666405fa05a294664d03492 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:31.213416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894feaf-d8df-4268-bc9a-ae5f8ddc89bc None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.213930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894feaf-d8df-4268-bc9a-ae5f8ddc89bc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.213930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894feaf-d8df-4268-bc9a-ae5f8ddc89bc None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.215760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 203/418] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:31.216828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:31.221719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.222008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.222274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.244807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:31.246147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:31.336834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:31.337007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 204/419] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:31.358640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f70328-1dfc-4d9b-a319-ff7197a0fdb0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:31.396462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:31.397112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.397391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.397718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.409215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:31.414347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:31.415266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:31.416040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 216/420] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:31.422481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:31.491322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:31.573962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2368c-9606-45f7-be6f-f61ad5e5700e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:31.628054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:31.628599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.628878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.629151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.629821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] RBAC: Adding query filter params (name=member) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:31.630122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:31.631527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8055e19-4fbb-4680-b9ba-8e8cb4ded935 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:31.646485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 205/421] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:31.651920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:31.689390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042da8-b23d-4b81-b468-c910da56ddc4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.689701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042da8-b23d-4b81-b468-c910da56ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.689926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042da8-b23d-4b81-b468-c910da56ddc4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.692114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 206/422] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:31.698508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.698799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.699078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.726900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:31.728217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:31.799007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 207/423] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:31.799954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:31.801624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688a673-433d-4ade-b78a-54985533ce02 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:31.841824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:31.842740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.843030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.843310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.846727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:31.877807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:31.958209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:31.959953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.009471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78272d03-1b1c-4f28-8c29-2f76c8e1825e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.071306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.072255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.072677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.073088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.086126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:32.091578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:32.092446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:32.093210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:32.095904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 208/424] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:32.101808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.121570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 217/425] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:32.128167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.194748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.223133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.230420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7f3a-dbd8-42be-9128-b2be28b86f29 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.249038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd949e-8220-4cee-9785-e90f7750e898 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.277477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.278248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.278669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.279114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.279879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:32.280255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.282361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.290605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.291510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.291930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.292342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.292526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 209/426] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:32.293512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.294252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.296004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.298682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 218/427] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:32.299771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.404261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.406923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae603-d282-465b-a7e6-1cfcecb3a094 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.412169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8f25b-370f-4d95-a536-4f5f7b68eb36 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.444714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.445476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.445757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.446032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.446976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.447601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.447880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.448149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.457303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:32.463159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:32.464049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:44:32.464848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:32.465445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 210/428] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:32.467102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.502843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.522043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.523844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.586781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04d770-a512-4f4e-8285-bc702836fd1e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.634634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.635308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.635587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.635862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.637870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.638866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.640720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.645960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 219/430] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:32.647159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.648451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 211/430] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:32.649456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.754195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:32.774481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870032e8-a369-4af3-b79e-9a6a14cdfdf1 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.790680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9b34b-0c9e-439c-8599-54966d98bdea None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:32.824940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.825893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.826184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.826510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.827263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:32.827577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.829032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.830268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:32.831023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.831488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.831971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.832907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] RBAC: Adding query filter params (name=member) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:32.833459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:32.835665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 212/431] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:32.835919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:32.839505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.857070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 220/432] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:32.876472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:32.988019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f326e-aed4-4a9f-a990-a9af59019f61 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.002791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.027593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.028149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.028428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.028703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] PATH_INFO: `/v3/projects/b7cf1c0e7a9b40d48890395c76780e60` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.029928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] RBAC: Authorizing `identity:get_project(project_id=b7cf1c0e7a9b40d48890395c76780e60)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.031426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.033080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 213/433] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/projects/b7cf1c0e7a9b40d48890395c76780e60 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:33.039090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.063327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.092499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383eb13-528e-4322-bade-35cda8819b88 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.130714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.131384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.131657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.131926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.157773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2f5a-b330-43f0-92de-7397f5cc614c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.223717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.225510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.227208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.227208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.227208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.228253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.229735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.231673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.231797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 214/434] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:33.239319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.301359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.343176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.349675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 221/435] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:33.357274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.378443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c33f4-9a09-4656-b7b3-31d00134d741 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.422804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.423506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.423795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.424069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.424957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=b7cf1c0e7a9b40d48890395c76780e60) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:33.425284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.430622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ee1bd-a8df-41d9-ac20-ea231a2163fe None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.442961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 215/436] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=b7cf1c0e7a9b40d48890395c76780e60&role.id=f12fcc11a666405fa05a294664d03492 => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:33.501163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a88c339-5c4b-46fd-bc32-f41b24f794d5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.549084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.550170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.550706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.551177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.553069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.554991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.557488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 222/437] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:33.558398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5bda-e7b6-46a7-881b-48c4b60b38d5 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.559253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5bda-e7b6-46a7-881b-48c4b60b38d5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.559253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d5bda-e7b6-46a7-881b-48c4b60b38d5 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.560669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 216/438] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:33.568145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.576020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247a262-2a75-4885-86a8-c0ca8045170b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.576366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247a262-2a75-4885-86a8-c0ca8045170b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.576572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247a262-2a75-4885-86a8-c0ca8045170b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.597745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e247a262-2a75-4885-86a8-c0ca8045170b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:33.599025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e247a262-2a75-4885-86a8-c0ca8045170b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:33.611108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.702194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 217/439] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:33.707220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10de76-4e88-4f70-a471-45d2171e8af1 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.707708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10de76-4e88-4f70-a471-45d2171e8af1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.708200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10de76-4e88-4f70-a471-45d2171e8af1 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.711079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 218/440] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:33.715806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487f60-63c4-4d22-986e-14c121faf237 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.716544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.716828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.717104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.743191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:33.744457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:33.775778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.775778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.775778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.775778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.776375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.778170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.780023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 223/441] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:33.788715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.848078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 219/442] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:33.924489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.929847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016c14b-6ee3-423f-a8b2-07e6d24c5f31 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.970598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.971241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.971520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.971793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.972551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=753c4dc548cd4aeabce829afb96fed52, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:33.972863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.974361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.981906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 224/443] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=753c4dc548cd4aeabce829afb96fed52 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:33.988648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.001630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.115192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.128070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ea462-02ce-4271-999c-3ce00ea28d53 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.169917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.170503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.170779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.171046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.184618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:34.189136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:34.190036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:34.190870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 220/444] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:34.196434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.252266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e337b-5508-4b7d-8b29-51ade5925971 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.277634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.295841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.296501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.296780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.297052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.309825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:34.315342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:34.316222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:34.316968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 225/445] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:34.322503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.328692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46281d9-a79f-42d0-b561-06c70a07b004 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.342629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.371596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.372130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.372406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.372757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.373433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:34.373739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.375194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.386336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 221/446] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:34.392890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.441465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73ef8b-ecfc-47b9-a60a-52c38095e40d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.475293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.475910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.476192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.476433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.477124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:34.477414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.478877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 226/447] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:34.495317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.496446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7836d75-68f6-4eab-8e3d-c2e5a7172728 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.525541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.532665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.533282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.533558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.533803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.577220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.596531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855e5a0-7c26-42de-8d69-b31439be582a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.607747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.609227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.630485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.631048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.631332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.631595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.702598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.704167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.709804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 222/448] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:44:34.715590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.745101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.786275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.806463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 227/449] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:34.813888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.819739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cb507-7d14-48a5-be0a-c512a7d787b7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.853623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.854249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.854552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.854826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] PATH_INFO: `/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.855954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] RBAC: Authorizing `identity:get_project(project_id=02e765af7d454a8f9fe205c67fe1fdbb)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.857467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.859151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 223/450] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/projects/02e765af7d454a8f9fe205c67fe1fdbb => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:34.864578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.915502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eb754-490c-4d08-97a8-64458be5fea2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.951470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.952167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.952867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.952867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] PATH_INFO: `/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.954062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] RBAC: Authorizing `identity:get_project(project_id=f7533d7a14534fcb9a3c1a9a17bbda8a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.955489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.957228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 228/451] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/projects/f7533d7a14534fcb9a3c1a9a17bbda8a => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:34.963412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.965105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.969432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa1d47-4843-4464-8a2c-7c90de258943 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:35.009445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:35.010100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:35.010399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:35.010675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] PATH_INFO: `/v3/groups/2a3e77a28e4f4a1084a57c89360c6592` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:35.012236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] RBAC: Authorizing `identity:get_group(group_id=2a3e77a28e4f4a1084a57c89360c6592)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:35.013729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:35.015376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 224/452] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/groups/2a3e77a28e4f4a1084a57c89360c6592 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:35.017582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:35.021975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:35.074250 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fd900-60af-4aa0-9a4d-7bd983ba4846 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:35.110389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:35.111061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:35.111333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:35.111622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] PATH_INFO: `/v3/groups/768771a0b3064733b184f4f6052255fc` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:35.112741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] RBAC: Authorizing `identity:get_group(group_id=768771a0b3064733b184f4f6052255fc)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:35.114247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:35.115846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 229/453] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/groups/768771a0b3064733b184f4f6052255fc => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:35.122000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:35.123238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431de87d-d701-471e-aaf5-d55f1b2b3a88 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:35.157722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:35.158421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:35.158708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:35.158977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:35.159706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] RBAC: Adding query filter params (group.id=2a3e77a28e4f4a1084a57c89360c6592, role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=02e765af7d454a8f9fe205c67fe1fdbb) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:35.160010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:35.161468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1ef7c-22e6-4651-a7d8-1245d8b9aa7b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:35.167457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 225/454] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:35 2022] GET /identity/v3/role_assignments?group.id=2a3e77a28e4f4a1084a57c89360c6592&scope.project.id=02e765af7d454a8f9fe205c67fe1fdbb&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:35.225777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34ff30-c712-4932-8076-23c0812c4e2f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:35.244711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:35.264221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:35.264915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:35.265195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:35.265464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:35.266212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] RBAC: Adding query filter params (group.id=768771a0b3064733b184f4f6052255fc, role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:35.266550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:35.268020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78dc2b-6704-4fd2-a1ea-67dd0ab93967 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:35.274639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 230/455] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Sep 13 19:44:35 2022] GET /identity/v3/role_assignments?group.id=768771a0b3064733b184f4f6052255fc&scope.project.id=f7533d7a14534fcb9a3c1a9a17bbda8a&role.id=f12fcc11a666405fa05a294664d03492 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:37.979329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51381-7077-4c3a-9299-3f00a4775129 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:37.979858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51381-7077-4c3a-9299-3f00a4775129 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:37.979858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51381-7077-4c3a-9299-3f00a4775129 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:37.981508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 226/456] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:37.987850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1a00-40bd-4b9d-ba1b-cb75b3676095 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:37.988384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1a00-40bd-4b9d-ba1b-cb75b3676095 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:37.988384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1a00-40bd-4b9d-ba1b-cb75b3676095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.009009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d30e1a00-40bd-4b9d-ba1b-cb75b3676095 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:38.010255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d30e1a00-40bd-4b9d-ba1b-cb75b3676095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:38.080606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 231/457] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:38.119113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1646ab-8bfb-44e6-bdb9-d653f3cc364c None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.119445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1646ab-8bfb-44e6-bdb9-d653f3cc364c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.119654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1646ab-8bfb-44e6-bdb9-d653f3cc364c None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.121001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 227/458] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:38.139361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.139657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.139924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.163131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:38.164388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:38.251892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 232/459] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:38.473289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1646ab-8bfb-44e6-bdb9-d653f3cc364c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:38.510833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:38.595385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1646ab-8bfb-44e6-bdb9-d653f3cc364c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:38.623130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:38.639200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:38.639851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.640126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.640395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.660319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:38.661851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:38.692113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:38.693080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:38.693887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:38.693887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:38.693887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:38.694538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 228/460] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:38.700893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:38.717659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:38.802754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:38.827168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f30b4-5965-4e71-87b0-f892ee5fec08 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:38.833572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ef9f4-f9ac-4b62-8276-59183179d2db None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:38.876085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:38.877021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.877457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.877729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.878381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:38.878693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:38.880068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e985e-310d-4cc0-a4de-5ab60e299ffe None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:38.880857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:38.882050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.883175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.883651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.894268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 229/461] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:38.896274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3871-9df0-4a2d-bd7b-c30db0456b98 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.896723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3871-9df0-4a2d-bd7b-c30db0456b98 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.896996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be3871-9df0-4a2d-bd7b-c30db0456b98 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.897643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:38.898600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 230/462] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:38.899292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be3871-9df0-4a2d-bd7b-c30db0456b98 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:38.899837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ec4724e7-20db-4a22-b944-27df1f594c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:38.921288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:38.922295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:38.923512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:38.923512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:38.923512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:38.926422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 233/463] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:38.929210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589f931-0ef8-4310-b150-171ce0d85924 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.929498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589f931-0ef8-4310-b150-171ce0d85924 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.929777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589f931-0ef8-4310-b150-171ce0d85924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.948821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4589f931-0ef8-4310-b150-171ce0d85924 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:38.950045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4589f931-0ef8-4310-b150-171ce0d85924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:39.021346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be3871-9df0-4a2d-bd7b-c30db0456b98 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.034397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 234/464] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:39.034397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab28ff0-8715-4b52-895e-36bd47f3a14d None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.034397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab28ff0-8715-4b52-895e-36bd47f3a14d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.034397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab28ff0-8715-4b52-895e-36bd47f3a14d None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.034397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 235/465] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:39.035230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab28ff0-8715-4b52-895e-36bd47f3a14d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.060854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.072883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.073507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.073783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.074050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.076173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.077735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3487f0c-982b-410d-a2d8-7a1382100de5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.098341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.111006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 231/466] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:38 2022] POST /identity/v3/users => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:39.115387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.115701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.118415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.161009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:39.164626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:39.212773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab28ff0-8715-4b52-895e-36bd47f3a14d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.288611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.289258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.289540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.290839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.291568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:39.291878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.293369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.310086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 232/467] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:39.321052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.324249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 236/468] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:39.331461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.425749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.448211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacc2a3-f825-4505-b99a-3ab1ea05511f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.466409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712809-f261-41de-b786-fe1dd3367d27 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.484977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.485630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.485908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.486176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.486881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.488608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.499252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.510804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.511946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.512638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.513135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.517775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 237/469] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:39 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:39.525742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.527873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:39.551562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:39.552534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:39.553352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:39.553352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:39.553352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:39.553352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 233/470] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:39.558999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.666045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.726220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca7a8a-791f-4433-bb4d-ba8c42cbfb86 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.783470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.784115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.784396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.784670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.785400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:39.785711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.787177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0a96d-0086-4897-b92d-48fd1c995d06 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.795985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.811654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 238/471] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:39.811654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cdb1-518f-49b6-9f14-27e28a7e3805 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.811654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cdb1-518f-49b6-9f14-27e28a7e3805 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.811654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209cdb1-518f-49b6-9f14-27e28a7e3805 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.813302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 239/472] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:39.819157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209cdb1-518f-49b6-9f14-27e28a7e3805 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.860113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841a811-3f8a-4bc7-a6b0-30fdba2244a2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.918242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.919430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.919909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.920385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.939465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:39.941501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8570c9b5-c183-4ef7-b5ae-37e59ebf080a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:39.963570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:39.965057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:39.966470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:39.966470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:39.966470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:39.967880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 234/473] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:39.973500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.973500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.973500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.977109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209cdb1-518f-49b6-9f14-27e28a7e3805 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.990937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:39.992931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:40.016380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:40.017307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.017748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.018164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.019034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:40.020759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:40.023911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:40.049135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 240/474] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:39 2022] POST /identity/v3/users => generated 249 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:40.055694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:40.075215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 235/475] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:40.155399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:40.222688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063cc8-71ba-42a2-bbef-57cff37a6fcd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:40.275753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:40.276451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.276739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.277013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.277714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:40.278025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:40.279687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:40.299288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 241/476] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:40.307619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:40.411640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:40.476042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3b509-66ed-41a5-a401-dc70cc2cb2ef None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:40.520232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:40.521198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.521647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.522086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.522935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:40.524849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:40.571717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 236/477] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:40 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:40.597322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d6e99-f1ec-4736-b206-4f86b3a04fd1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:40.617431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:40.650725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:40.651745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.652282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.652723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.653661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:40.655365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:40.665010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.sql.core [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:44:40.665010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:44:40.665010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [parameters: {'id': 'f38b88bc8bdd46299c765604129bb7c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:44:40.665010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result.read() Sep 13 19:44:40.684044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:44:40.685049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application yield resource Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise self.value Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application session.commit() Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:44:40.686095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:44:40.687072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:44:40.688069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result.read() Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:44:40.689014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [parameters: {'id': 'f38b88bc8bdd46299c765604129bb7c9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:40.689874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:44:40.690907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:40.693861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 242/478] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Sep 13 19:44:40 2022] POST /identity/v3/roles => generated 144 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:40.702813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:40.736009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:40.841916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7020-b537-4260-8c1e-2b1384dabe9d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:40.881800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:40.882497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.882779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.883054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.894384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c27b5629-9e15-4f19-8f18-70e68436a106 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:40.899956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:40.901952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:40.903906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 237/479] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:44:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:40.912476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:41.026137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e5a80-519a-4ad0-9149-da3d1bbd5570 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:41.062147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:41.063479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:41.063921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:41.064326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:41.065133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:41.065565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:41.067174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964a272-ac7b-4869-af52-7528dd4d1ce6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:41.076882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 243/480] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:41.963027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5df2-3a1c-4a01-911e-7dde045189e1 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:41.963565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5df2-3a1c-4a01-911e-7dde045189e1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:41.963854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5df2-3a1c-4a01-911e-7dde045189e1 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:41.965722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 238/481] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:41.971739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572c6bf-3b3f-4879-bd5a-00eaa03ce6a3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:41.972105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572c6bf-3b3f-4879-bd5a-00eaa03ce6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:41.972390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572c6bf-3b3f-4879-bd5a-00eaa03ce6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:41.999414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8572c6bf-3b3f-4879-bd5a-00eaa03ce6a3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:42.000641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8572c6bf-3b3f-4879-bd5a-00eaa03ce6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:42.037285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:42.080985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 244/482] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:42.498493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d5df2-3a1c-4a01-911e-7dde045189e1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:42.512376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175798-0efd-4204-a45f-a26a8bceb147 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.512718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175798-0efd-4204-a45f-a26a8bceb147 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.513044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175798-0efd-4204-a45f-a26a8bceb147 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.517115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 245/483] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:42.519869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.520162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.520434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.556279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:42.557603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:42.647015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:42.676195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d5df2-3a1c-4a01-911e-7dde045189e1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:42.695875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 246/484] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:42.722203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:42.722888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.723169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.723443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.740803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-454b3602-1955-43b1-b0d8-994541eb1be8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:42.751058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:42.752171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:42.753123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 239/485] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:42.758821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:42.865028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16f0f-23c2-4055-8bc6-871fdce30edc None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.865426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16f0f-23c2-4055-8bc6-871fdce30edc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.865599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16f0f-23c2-4055-8bc6-871fdce30edc None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.867346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 240/486] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:42.880948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.881226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.881500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.906006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:42.907249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:42.913112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:42.916126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71c2bc-3e09-421f-a913-5a97c15c6b74 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:42.954745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:42.955380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:42.955666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:42.955943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:42.956628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:42.956933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:42.958509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:42.973859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 247/487] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:42.975160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.058612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 241/488] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:43.100399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.142386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f75316-dbe8-4449-9498-fff95373b765 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.176304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.176960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.177245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.177522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.188903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:43.190532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:43.205588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:43.206555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:43.207361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:43.207361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:43.207361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:43.207361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 248/489] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:43.212793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.217462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.266656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.284706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666390e-4b97-4f25-ac90-d53bb11d8fee None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.360235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.360884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.361164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.361435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.367996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bb6f3-6b70-477a-9782-f121c59f9541 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.391226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:43.397646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:43.398568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:43.399345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:43.401329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 242/490] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:43.416570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.452839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.453482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.453765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.454040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.454745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:43.455922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:43.457779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:43.476212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 249/491] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:43.478063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.563517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9484d4b-6311-4bf0-91aa-e26d1aafa09d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.595279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.603058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.614093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.615101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.615603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.616044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.617293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:43.617832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:43.619525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:43.631853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 243/492] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:43.633245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.634932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d249f0-8393-4a06-aedf-5e0108517965 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.684120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.684803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.685089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.685338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.698556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:43.704549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:43.705616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:43.706459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:43.710810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 250/493] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:43.712105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.769697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d3435-a401-4861-9a0d-cf884af8d8af None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.808521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.809165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.809866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.809866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.819781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:43.821078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:43.828667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814064ed-03ba-4ab9-998e-a82f4a2cccaa None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-4e7b22c7-b0b3-4468-87d0-14a37bbdb284 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:43.834627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:43.835512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:43.836365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 244/494] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:43.837290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8fd5c-28b8-494e-9e05-5c6f770defd4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.837556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8fd5c-28b8-494e-9e05-5c6f770defd4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.837823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8fd5c-28b8-494e-9e05-5c6f770defd4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.839079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 245/495] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:43.839751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8fd5c-28b8-494e-9e05-5c6f770defd4 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.874924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:43.875703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.876160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.876510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.878849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.891979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:43.893525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:43.896863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-fee8a211-4af6-4399-8422-c45d777ba496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:43.910737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:43.912030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 251/496] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dae0-6d35-4f15-a9f3-8b42b09d5f7f None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dae0-6d35-4f15-a9f3-8b42b09d5f7f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.913627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687dae0-6d35-4f15-a9f3-8b42b09d5f7f None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.915061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 252/497] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:43.915483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687dae0-6d35-4f15-a9f3-8b42b09d5f7f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:43.960669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8fd5c-28b8-494e-9e05-5c6f770defd4 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.000272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.000905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.001177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.001586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.002755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:44.003254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.005164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d68d5-ef02-41c1-9683-c5339b1417a4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.015807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 246/498] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:44.017789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48b86-7eca-44c3-8479-ff1142fa9de0 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.018248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48b86-7eca-44c3-8479-ff1142fa9de0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.018716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48b86-7eca-44c3-8479-ff1142fa9de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.032875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687dae0-6d35-4f15-a9f3-8b42b09d5f7f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.041606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bde48b86-7eca-44c3-8479-ff1142fa9de0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:44.043782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bde48b86-7eca-44c3-8479-ff1142fa9de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:44.082832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.083450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.083727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.083995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.084732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:44.085030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.086522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.125493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 253/499] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:44:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:44.126271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.142942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 247/500] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:44.144204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.144487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.144762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.172244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:44.173536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:44.184586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.220496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.252281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e8890-30c1-4d8c-ae8d-2dcb63c5dacd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.266004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 248/501] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:44.266004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.295871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.296862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.297209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.297548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.298549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:44.299364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.301440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.316841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 254/502] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:44.318457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.381720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235d2b4-457c-4d91-8f51-115d558f7e3e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.420008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.420662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.420949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.421234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.433583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.435334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:44.451003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:44.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:44.453097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:44.453097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:44.453097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:44.453097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 249/503] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:44.453097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.461849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed47c8f-657e-4722-a4c1-5fa9057efb58 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.465318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.484428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.507997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.508877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.509120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.509415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.573384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba96dcc-9743-4193-82a1-5065c1062c59 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.585226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.586753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.606240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.606965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.607243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.607537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.615317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.617145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:44.628496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:44.629522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 250/504] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:44.630612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.686908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 255/505] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:44.687682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.693443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.718605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.734360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a6982-ae32-406c-ab72-543fc8d241fb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.767447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.768085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.768354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.768623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.779244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:44.784641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:44.785498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:44.786272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 251/506] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:44.786941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.790950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c573a-f1c3-4f3f-9a22-27d7f5b7cff5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.824240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.824867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.825143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.825577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.826093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:44.826422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.827861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.840080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 256/507] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:44.841073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.893631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f03bbe-823b-4bcf-915e-da60060fb466 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.904484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.931713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.932356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.932633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.932906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.944630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebfd14-fcfe-4409-897d-583a2f7f4443 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.945305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.946037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.948912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:44.963479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:44.964466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:44.965255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:44.965255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:44.965255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:44.965255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 252/508] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:44.965493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.984074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.984694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.984991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.985265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.986017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:44.986394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.987827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.006192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 257/509] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:45.007070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.074327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b1ac-7e5e-4968-af54-8247fcfdf217 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.109835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.110266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c41cbc-3337-4120-bec0-2861493e6daf None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.110511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.110770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.111037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.122057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.123574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.134274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:45.138302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:45.139187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:45.139989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:45.139989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:45.139989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:45.139989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 253/510] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:45.140242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.152293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.152917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.153203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.153514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.154181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:45.154515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.155926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.164495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 258/511] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:45.165314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.198853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.255016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4539-abd3-4ad5-aaea-0594c5cffe25 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.271061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0d7a5-4e18-4585-9c33-bdeb1889d917 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.288852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.289503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.289781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.290059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.298082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.299629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.306250 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.306917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.307197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.307469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.308124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:45.308440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.309862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:45.311235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:45.312078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:45.312938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 254/512] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:45.313464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.325321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 259/513] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:45.326202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.375189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.421252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27ed23-5d9d-4913-95c3-c3d9f9261a6d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.430734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.431976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c8722-a933-461e-a9ae-0cf0a2f17d27 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.457364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.458003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.458285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.458615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.466020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.466656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.466932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.467196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.467834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:45.468127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.469556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.481197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 260/514] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:45.482059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.533755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.535303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.594322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5646bf8-9be1-4d64-9470-e17c128db925 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.596616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.630324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.630997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.631276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.631547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.643036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 255/515] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:45.643036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.647763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.649272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.650700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:45.666083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:45.667041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:45.667852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:45.667852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:45.667852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:45.672677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 261/516] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:45.673633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.766912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3552-a422-4351-8109-5c869f9e91ba None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.791045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e0119-e0a9-4008-9c5d-f5f83f7be77a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.804716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.805510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.805787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.806390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.806836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:45.807156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.808587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.826354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:45.827029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:45.827306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:45.827543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:45.828197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:45.829682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:45.850315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 256/517] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:45.851276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.852159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 262/518] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:44:45 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:44:45.852990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:45.872947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.939615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:45.989064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a843cb-53e1-4789-92a8-7545222f96ac None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:45.996121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12a49-534b-4a20-8965-ae5516428b4c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.029018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.029585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.029855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.030123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.036342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.037247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.037580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.037850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.038266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.039112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:46.051507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:46.052393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:46.053204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 257/519] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:46.053655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.053725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.054578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:46.072411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:46.073326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:46.074146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:46.074146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:46.074146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:46.074146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 263/520] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:46.074801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.146955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.177358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bc467-7ec1-4891-aa33-e42c9d896cea None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.189529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7240-2e0f-4c33-ba49-1bcd326c2ea7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.205340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.215565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.216469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.216729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.217010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.217661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:46.217962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.219403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.225973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.226883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.227158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.227548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.236314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 258/521] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:46.236314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.305128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.306587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.345253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36471dc2-0820-42bd-af22-7e5d1cfd8f52 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.371720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.383115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.383739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.384014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.384281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.384978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:46.385280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.386749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.407891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 259/522] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:46.408630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.419886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 264/523] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:46.420937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.465348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.528030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c9633-16ce-476c-b83d-44408a7b7920 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.532197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2763d1-d073-4743-ae7c-d8baebb85266 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.572235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.572846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.572953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.573121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.573393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.573484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.573750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.574023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.574113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:46.574340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.575866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.583012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.585173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.593570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 260/524] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:46.594463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:46.599913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:46.600806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:44:46.601774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:46.605956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 265/525] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:46.607021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.635038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.703654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3a2dd-7f46-4c23-a9cd-a601798a2a60 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.712472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.713503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd126a0-65e1-4c15-b313-177786451c88 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.748906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.749529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.749805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.750071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.750849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:46.751144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.752266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.752591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.753213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.753651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.754059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.758339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 261/526] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:46.759169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.772166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.774139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:46.792384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:46.793673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:46.794767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:46.794767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:46.794767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:46.796797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 266/527] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:46.798067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:46.893306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be046b0-ecef-4bbf-9afb-bd5e590567ba None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.906239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:46.933007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417c083-e72a-475e-83f5-12b28e23b5d3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:46.943101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.944198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.944680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.945282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.955010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.957087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.971795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc268b2-0678-4217-af49-027282df077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:46.972697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc268b2-0678-4217-af49-027282df077f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.972996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc268b2-0678-4217-af49-027282df077f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:46.973380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:46.974171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:46.975050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:46.975452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc268b2-0678-4217-af49-027282df077f None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.975452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc268b2-0678-4217-af49-027282df077f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:46.975452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc268b2-0678-4217-af49-027282df077f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:46.978471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc268b2-0678-4217-af49-027282df077f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:46.982547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 262/528] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:46.984642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.005098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 267/529] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.006292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc268b2-0678-4217-af49-027282df077f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.018261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.095829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10ca71-2895-4a8f-a0dd-753aa14aea62 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.125092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc268b2-0678-4217-af49-027282df077f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.141642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.142855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.143651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.144179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.145651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:47.146234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.148401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.164237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.164972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 263/530] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.165233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.165718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.166113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.166212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.171102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:47.171420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.172909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.174900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.196308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 268/531] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.197668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.283842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.296774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dee60-3b1e-4746-85c6-b02a3c8ab0e9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.333696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.334714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.335193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.335664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.336485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7b8ed-bd00-4e84-98ec-bcc4c3be9589 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.386090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.386825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.387111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.387382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.396969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.398808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.412703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.414528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:47.414965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:47.415789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:47.416627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:47.421178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 269/532] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:47.422826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.500089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.518336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.531489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 264/533] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:47.532544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.561926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e515eaf-03a4-48a9-824e-140c3f7f3a79 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.612113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.613150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.613633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.614353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.615958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=0d318e084073402e8d3385eb69c9685b) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:47.616482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.619035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.627310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 270/534] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/role_assignments?user.id=0d318e084073402e8d3385eb69c9685b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.628472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.685127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c6d3d-26dc-42a5-83ee-5fca57588d7a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.747337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.748554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.749043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.749529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.751020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:47.751605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.753467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.767952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 265/535] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.774496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.798926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97b87-bea1-4ace-b6c2-a8f34da4d507 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.841590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.845546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.887227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.887978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.888267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.888546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.907168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac591e1-e7ed-4e21-a370-05a825fc6152 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.919956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.921156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.947649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.948757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.949103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.949531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.950407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=1a83958d50934b24ae40da8760b2fb85) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:47.950742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.952243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.958320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 266/536] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/role_assignments?user.id=1a83958d50934b24ae40da8760b2fb85&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:47.959868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:47.960805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:47.961611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:47.961611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:47.961611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:47.961611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 271/537] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:47.967196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:48.107984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab433d13-4377-4700-9298-aeed50335176 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:48.124915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:48.160863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:48.161887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.162362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.162890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.164106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:48.164642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:48.167146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c3555-01f0-407d-a397-6b8866ad5752 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:48.194649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 267/538] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:48.199962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:48.212657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025bf0-5a02-48b4-9a5d-f6de05913cc3 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.213233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025bf0-5a02-48b4-9a5d-f6de05913cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.213696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025bf0-5a02-48b4-9a5d-f6de05913cc3 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.216315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 268/539] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:48.223065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.223520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.223977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.271215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:48.272768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:48.379134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a04892-22d1-4f05-aa25-408d2b844cb1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:48.417681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:48.418761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.419295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.419753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.427398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:48.429145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:48.440581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 269/540] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:48.441925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:48.447429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:48.448254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:48.450550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 272/541] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:48.463465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:48.620177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bffd3-62f0-4d63-a47c-8d93004393bb None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:48.623563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:48.657280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:48.657899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.658175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.658470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.659173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:48.659475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:48.660977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:48.673762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 270/542] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:48.679521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:48.734965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:48.824748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cd0e6-ad2b-4c50-a840-7089b9f29631 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:48.826683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:48.875572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:48.875572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:48.875572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:48.875572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:48.876199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=f641d3de0a3a43ad8bc789483ffd5a72) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:48.876199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:48.878299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:48.886084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 273/543] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:48 2022] GET /identity/v3/role_assignments?user.id=f641d3de0a3a43ad8bc789483ffd5a72&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:49.040531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05aa75-14f8-41dc-b379-1aa712dba0aa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:49.093928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:49.094618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.094896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.095175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.111007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-3acf2879-901f-45aa-be5e-25a6ad0858b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:49.116371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:49.117610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:49.118829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 271/544] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:49.126146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.204326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe3af-dac7-49cc-8b84-9a0597e29f63 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.204727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe3af-dac7-49cc-8b84-9a0597e29f63 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.205028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fe3af-dac7-49cc-8b84-9a0597e29f63 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.206466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 272/545] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:49.214251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.214584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.214820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.273214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:49.274478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:49.297716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:49.311240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:49.313619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7205390-e449-4e77-b7a6-53ca4233d5a5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:49.383531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:49.384865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.385315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.385761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.386636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:49.387097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:49.388716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:49.407944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 274/546] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:49.410863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.413186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 273/547] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:49.519664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a5a8c-5f9d-4cb1-b533-5efa2cde46ef None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:49.580107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:49.580862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.581143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.581421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.598731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:49.600274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:49.622737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:49.623892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:49.625426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:49.625426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:49.625426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:49.625426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 275/548] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:49.630059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.769424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c695f-ca97-46f0-ba90-37b2d19d9092 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:49.796953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.810122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:49.824014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:49.825160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.825640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.826146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.827786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:49.828411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:49.830185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:49.858335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 274/549] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:49.863620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.892755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:49.968458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee53af0-f12a-4122-9129-b88d7e6fe013 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.020894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.021550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.021860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.022076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.037826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:50.043486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:50.044678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:50.045448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:50.046058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 276/550] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:50.052113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.112656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d322c-b346-44b2-b06f-2a12a81c2b1b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.130801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.187678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95252bc4-486f-4d8f-a2e3-b73c06b97094 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.191167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.191821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.192143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.192413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.212217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.213705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:50.235998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:50.237067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:44:50.238050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:50.238599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.238599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.238599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.238813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 275/551] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:50.238878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.238878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:50.238878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.242389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.243324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.250329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 277/552] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:50.256902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.388378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.418054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83377-e326-4bc9-9334-bd9edc2e513d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.424895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1672107-5049-4fd4-845e-c0bac3b7761b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.459181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.460186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.460673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.461156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.462087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:50.462670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.464301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.492194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.496438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 276/553] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:50.499092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.499760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.500041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.500311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.503402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.527336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.528908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:50.564052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:50.565098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:50.565926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:50.565926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:50.565926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:50.569494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 278/554] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:50.574523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.671162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00825-d215-4b07-97f5-c91d16286be9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.704265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.709656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.711021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.711585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.711907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.717493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a79a4-6aa9-449c-8d3b-435076fcc90f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.756670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.757286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.757559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.757821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.758585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:50.758818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.760158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d698cc-851d-434f-be17-891c90465b65 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 279/555] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597bd1-7308-4c92-8d85-8431f31f4bdc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597bd1-7308-4c92-8d85-8431f31f4bdc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597bd1-7308-4c92-8d85-8431f31f4bdc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 280/556] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:50.775845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836502db-5fd9-46ab-9403-d2f048370acb None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.776346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836502db-5fd9-46ab-9403-d2f048370acb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.776346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836502db-5fd9-46ab-9403-d2f048370acb None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.777606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 281/557] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:50.779792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836502db-5fd9-46ab-9403-d2f048370acb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:50.795823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.797300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396bf33-c5c5-415f-ae42-50e2b0f0d88b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.828461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.890108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836502db-5fd9-46ab-9403-d2f048370acb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.911625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 277/558] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:50.912916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.913197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.913468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.930520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.931168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.931374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.931647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.931863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:50.933062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:50.939454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.940945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-beddc2b5-e629-440b-8fab-8fadae163199 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:50.953453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:50.954515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:50.955446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 282/559] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:44:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:50.956963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.956963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.956963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.965392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.980220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:50.981766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:51.017860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 278/560] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:51.018744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.076771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 283/561] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:51.076771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.142260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.177916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a8be0-ae95-4dd5-b64b-a3ac5467ad46 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.222779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.222779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.222779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.222779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.229125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.230678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:51.246416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:51.247817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:51.249298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:51.249298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:51.249298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:51.249298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 279/562] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:51.253457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.279817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.303970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3b6f2-c179-495d-abc2-0aceea15c43d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.352103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.352941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.353226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.353813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.354859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:51.355168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.357334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.365592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e0cf8-eaa9-4b90-926f-746dfc5c4a3d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.385481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 284/563] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:51.387159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.410976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.411598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.411876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.412145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.412891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:51.413266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.414849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.427276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 280/564] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:51.428098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.503630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.509583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.514720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96bfb5-82c1-496d-9924-2cd0e346a423 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.538382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756b581-ae2b-4f3d-b98c-85e6b60768b7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.549712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.550377 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.550665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.550937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.561560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:51.568599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:51.569536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 285/565] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:51.570360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.578085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.578771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.579049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.579316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.590873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.592387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:51.607098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:51.608093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:51.614247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:51.614247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:51.614247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:51.614247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 281/566] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:51.614247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.678049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8487-5b69-4a06-a287-91b3b387eb2e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.714770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.715378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.715715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.715998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.718273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42751dbe-3715-479a-a432-700ea9aee2b0 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.726909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a407d18f-43b3-4f8b-acc8-480ca0b6dab3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:51.731852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:51.732877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:51.733730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 286/567] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:51.734443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510afc5-bc01-4ce7-b3ca-0fdd433b2aa5 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.734443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510afc5-bc01-4ce7-b3ca-0fdd433b2aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.734755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510afc5-bc01-4ce7-b3ca-0fdd433b2aa5 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.735983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 287/568] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:51.736671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510afc5-bc01-4ce7-b3ca-0fdd433b2aa5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.751579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.751795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:51.761870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.762589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.762878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.763618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.770541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.772006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:51.784246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:51.785166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:51.786023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 282/569] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:51.787012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.848353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510afc5-bc01-4ce7-b3ca-0fdd433b2aa5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.883992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.884612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.884888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.885395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.885816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:51.886117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.887647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.890952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee065f6f-3cfc-4aaa-8753-9543e0dd18fe None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:51.895769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 288/570] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:51.896895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:51.925238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:51.925864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.926126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.926421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.927061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:51.927362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:51.928726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401c3b2-7204-402b-a080-3fc2f1a27c17 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:51.940396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 283/571] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:51.941567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:51.941841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:51.942110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:51.959732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:51.960915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:51.969789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.001895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6acb48-de77-4054-8b08-3662cdb09026 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.010005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.035470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 284/572] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:52.036237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.048055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.048769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.049056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.049316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.050075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.050299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.051958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.061194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 289/573] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.062230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.152847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ce3a-a3f6-4ce1-b419-eeeeb8f91c29 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.171688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27867-a329-4cf0-8ea1-a5a91c6cc1d4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.191790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.192452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.192752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.193029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.193741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.194014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.195984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.208001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.208644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.208925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.209199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.215867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 285/574] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.215867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.216689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.227211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.228693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:52.244727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:52.250731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:52.251830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:52.251830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:52.251830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:52.251830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 290/575] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:52.251830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.262171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.331140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3b87a-6e26-47f3-adb0-ccb1499818f7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.352999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae40af-79e3-4666-ad92-d8a486478172 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.366806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.367451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.367727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.367994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.374662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.376117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:52.387349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:52.388532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:52.389415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:52.389851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.389851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.389851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.390061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 286/576] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:52.390129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.390194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.402089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.403676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:52.418242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:52.419414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:52.420268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:52.420268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:52.420268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:52.420268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 291/577] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:52.420268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.471191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.487941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.495282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4bc4-5f37-4b65-a567-c32266af80fb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.530405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.531041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.531296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.531584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.532594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=09cae6567bc540bab88dc59c1ddee27b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.532594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.534037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.537177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816f029-a44b-4b2b-90c8-763c535f1a7b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.540888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 287/578] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/role_assignments?user.id=09cae6567bc540bab88dc59c1ddee27b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.541375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.581965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.582713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.582985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.583269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.584187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.584540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.586143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.599409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 292/579] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.604316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.667222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72db12b-5e5f-459f-8d98-db28f0bff7c6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.710662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.711437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.711721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.711973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.712671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.717652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5440f0b-bec6-493f-aad9-bbccc7e78734 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.725330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:52.732613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:52.733706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:52.734762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:52.738602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 288/580] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:52.738602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.761806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:52.762928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.763919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.763919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.764217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.764863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.765200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.766655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.783755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 293/581] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.783755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.855470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c2e0e-e650-4679-b7c7-1796d690d3ac None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.885095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24d8d2-88ab-4274-8c26-22a7c2e04488 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:52.892777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.893984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.894264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.894573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.895673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:52.895980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.898067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.913254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 289/582] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:52.914125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.927393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:52.928041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.928347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.928654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] PATH_INFO: `/v3/users/glance` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.937939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:52.940191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:52.953451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:52.954419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:52.955349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:52.959481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 294/583] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:52.960219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.981189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.007489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.030557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279de1c-2ed0-4958-b545-41468946a684 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.067047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f75c-7b73-4d2f-9576-e04b335276ee None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.071448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.072070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.072343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.072608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.073281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:53.073685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.074999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.090175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 290/584] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:53.090175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.125058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.125651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.125933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.126209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.143669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=463fcb701698473bad12e8101b9018ce, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.145117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.161369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 295/585] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:52 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:53.161369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.209189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e676f8-b7d7-4f54-af74-af9ec031de55 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.248733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.249546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.249928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.250285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.258712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.259635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.261109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:53.274403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:53.275418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:53.276330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 291/586] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:53.276912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.280181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.307037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e245f0-0b16-424c-b8da-316830df40da None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.349047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.350019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.350592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.351043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.352077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:53.352565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.354299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.375903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 296/587] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:53.377073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.394829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fae1e-bf8f-4585-a296-ba5de8c5da48 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.447544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.448300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.448648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.449117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.464028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.466938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:53.487702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:53.488685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:53.489542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:53.489542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.489542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:53.489542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 292/588] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:53.490715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.505915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab392e-4ad5-4fd0-994f-2191ec5a87f3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.533567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.550670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.551465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.551819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.552160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.552993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:53.553361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.555078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.566145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.585444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 297/589] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:53.586526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.631649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000f5c6-97a7-49cf-8f31-df51436285a9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.671486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.672136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.672417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.672769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.685765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.687753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:53.704010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:53.704977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:53.706445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:53.706445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.706445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:53.706445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 293/590] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:53.709623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5630c24-a576-44d1-8d88-7a00306aaef1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.710644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.755630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.756796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.757082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.757358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.758487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:53.758802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.760908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.776898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 298/591] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:53.778065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.834162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a02ac-7262-427d-8f28-7babc5f89849 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.843596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.853788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.899391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.899935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.900218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.900498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.914809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.918690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.932636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3832-7864-4183-8bc2-abdbab8d1b6b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:53.958448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:53.960249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:53.962007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:53.962007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:53.962007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:53.970792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 294/592] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:53.970792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:53.990413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.991056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.991689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.991965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.992600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:53.992902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.994313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.014523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 299/593] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:54.015463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.114176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c45ac1-5cf6-4375-b0d3-a73c58f1f1c5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.118762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778cc797-dce3-47d3-afbc-d72c788a386c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.122508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.161421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.162153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.162581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.163003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.163740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:54.164080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.164269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.165025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.165447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.165857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.166351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.176411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.178408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.183889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 300/594] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:54.183889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:54.197332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:54.198329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:44:54.199283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:54.204729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 295/595] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:54.205644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.226823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.310842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dbf83-29d8-43cc-ad97-76f8adf52a1b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.319109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3468-4e6d-4fa8-bffe-082b86cd8713 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.348586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.349224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.349535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.349769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.357449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.359306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.364871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.366080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.366935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.367431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:54.371802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:54.372649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:54.373502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:54.377718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 301/596] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:54.378087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.378713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.380149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:54.395792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:54.396725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:54.397690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:54.405109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 296/597] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:54.405109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.413942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.499529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e872a-ffa0-4c52-abf5-5beb5e3ee7ec None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.506658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.535179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b43eb-cd89-4c03-a55c-26a8d3f08619 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.545787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.546465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.547047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.547153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.547818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:54.548124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.549674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.570539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 302/598] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:54.577859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.578909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.579421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.579903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.580371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.581250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:54.582038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.585404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.601654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 297/599] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:54.602885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.681324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.702082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af6ead-658b-4135-9939-7f9179edb72d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.738485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.739135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.739382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.739713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.740343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:54.740648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.742513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.742755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49aba8-7030-4a7a-b0c9-648d8b334e1d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.755593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 303/600] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:54.756509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.785959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.786660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.786940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.787213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.794720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.798848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.800645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-b7071d7b-d544-49ee-9166-59853b944c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:54.816581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:54.817737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:54.818645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:54.818645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:54.818645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:54.822558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 298/601] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:54.823947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc045-9d71-4658-bdb0-32253606b8c8 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.824228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc045-9d71-4658-bdb0-32253606b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.824495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbc045-9d71-4658-bdb0-32253606b8c8 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.825875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 299/602] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:54.826578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbc045-9d71-4658-bdb0-32253606b8c8 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:54.872315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb1cb0-34d9-433a-88fa-dce5563c36b7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.908744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:54.909421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:54.909705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:54.910723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/0d318e084073402e8d3385eb69c9685b/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:54.923046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:54.927078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=0d318e084073402e8d3385eb69c9685b, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:54.929172 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:54.954557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 304/603] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:54 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/0d318e084073402e8d3385eb69c9685b/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:54.954992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbc045-9d71-4658-bdb0-32253606b8c8 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:54.966399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:55.011511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.012483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.012870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.013255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/1a83958d50934b24ae40da8760b2fb85/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.034683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=1a83958d50934b24ae40da8760b2fb85, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:55.036282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f630db-4a43-4b90-b23f-580981713942 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:55.065326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 300/604] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:54 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/1a83958d50934b24ae40da8760b2fb85/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:55.067054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.067403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.067743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.106213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a94688-9de6-43cd-a51d-b1b2a5fd38ae None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:55.113448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:55.114808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:55.149687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.150323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.150630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.150899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.151552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:55.151944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:55.153429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:55.168953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 305/605] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:55.175269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:55.182166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:55.243855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 301/606] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:55.254468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:55.310994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d9e4f-1888-40cf-ae66-831d97657150 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:55.353391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.354055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.354330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.354631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.362512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:55.364054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ed7660d2-83b3-423f-9687-4acff3aa50b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:55.376623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:55.377541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:55.378445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 306/607] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:55.385138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:55.454914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959f3c-c08b-4d1b-bcc8-b88a8a490113 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.455236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959f3c-c08b-4d1b-bcc8-b88a8a490113 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.455464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86959f3c-c08b-4d1b-bcc8-b88a8a490113 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.457092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 307/608] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:55.463181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.463468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.463846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.500062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:55.503229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:55.621920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0d6f6-c3bd-4c48-9bac-af6924c72227 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:55.657487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.658505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.658613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 308/609] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:44:55.659413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.659904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.660012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:55.660886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:55.661402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:55.663057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:55.676618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 302/610] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:55.682769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:55.746902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:55.756112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:55.890313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f4487-eda3-4c6c-94bf-d5faafb61283 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:55.901277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2239-e913-4a8b-9e16-604ce2c3b49c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:55.986809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.987469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.987744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.988009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:55.997118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:55.997908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:55.998043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:55.998388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/f641d3de0a3a43ad8bc789483ffd5a72/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.012751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:56.017571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:56.019352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:56.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 309/611] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:56.027375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=f641d3de0a3a43ad8bc789483ffd5a72, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.029132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.031429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.052979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 303/612] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:55 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/f641d3de0a3a43ad8bc789483ffd5a72/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:44:56.175015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f702946-c029-4c90-adbf-62ea1e033b3e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.176566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:56.181135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.242828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.243965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.244441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.244899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.245884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:56.246433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.248215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.264009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 310/613] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:56.271895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.342765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:56.356285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0f9a1-e75f-4977-bb84-4ffdd8984dfa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.399776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.400515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.400515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.400776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.417073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:56.423658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:56.424559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:56.425396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 304/614] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:56.431115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.472271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9dbc7-91e7-4724-90c9-66b3feb3df9e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.543092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0072be9-f2a1-4d73-8fc3-9991ed134fdb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.543305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.543836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.544121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.544392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.580961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.581787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.582206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.582642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.583475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:56.583918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.585513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.587271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.588763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.595539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 305/615] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:56.605962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:56.609687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:56.610722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:56.611591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:56.611591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:56.611591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:56.611591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 311/616] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:56.616692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:56.619216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.656074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:56.720459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9071e34-9b7a-4ce3-9b7f-35e97dc40ce4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.756786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.757782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.758236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.758701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.770434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.772234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:56.787820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:56.788812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:56.789823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:56.789823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:56.789823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:56.789823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 306/617] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:56.794649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.844094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e47ce-22df-4d24-a652-0d3006ec8469 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:56.900534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:56.901299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:56.902407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:56.902701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:56.903036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:56.903729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:56.906388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:56.906388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:56.922796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 312/618] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:56.930596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:56.990092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad87a0-1200-46f4-aca2-9a2476105736 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.064267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.075323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.076014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.076292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.076568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.077240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:57.077541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.078951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.091951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fd0e5-ceba-4701-8d41-b41dfab1fa1d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.113357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 307/619] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:57.123608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.140556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.141567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.142035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.142632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.150899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.152647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:57.165321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:57.166309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:44:57.170095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:57.177624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 313/620] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:44:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:57.179215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.284897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2981449-8eac-4f42-900f-58af40b97a65 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.319376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46237893-d762-4956-b107-5a222145477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.320224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46237893-d762-4956-b107-5a222145477f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.320676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46237893-d762-4956-b107-5a222145477f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.321094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46237893-d762-4956-b107-5a222145477f None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.321958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46237893-d762-4956-b107-5a222145477f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:57.322437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46237893-d762-4956-b107-5a222145477f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.324230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46237893-d762-4956-b107-5a222145477f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.339331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.346528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 314/621] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:44:57.348965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46237893-d762-4956-b107-5a222145477f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.438706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fc54f-edf5-4ac0-89fe-553cf39eb929 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.457819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.483141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.483853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.484128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.484442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.493702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.495370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:57.510224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:57.511243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:57.512262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 308/622] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:57.512838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46237893-d762-4956-b107-5a222145477f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.524419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.583370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.584025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.584358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.584628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.608965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.610506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-eb619060-715d-4aa2-801e-4bb58581262a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:57.630949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:57.632160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:57.633234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:57.633234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:57.633234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:57.637251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 315/623] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:44:57.639227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceada2-8ab1-4e3c-afa2-3672203600ee None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.639628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceada2-8ab1-4e3c-afa2-3672203600ee None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.639907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ceada2-8ab1-4e3c-afa2-3672203600ee None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.642195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 316/624] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:57.643017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ceada2-8ab1-4e3c-afa2-3672203600ee None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.692414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baab79a-ed77-44f5-b32a-509a635cf23a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.704649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.737209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.737885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.738096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.738446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.739835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:57.739835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.741040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586689e5-28d0-4556-be5a-b658b7112383 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.765602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 309/625] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:57.767550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda55d-07a8-41db-96b0-6c282d77d759 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.767942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda55d-07a8-41db-96b0-6c282d77d759 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.768216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cda55d-07a8-41db-96b0-6c282d77d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.777535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ceada2-8ab1-4e3c-afa2-3672203600ee None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.787435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-51cda55d-07a8-41db-96b0-6c282d77d759 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:57.788648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-51cda55d-07a8-41db-96b0-6c282d77d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:57.814895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:57.815780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.816229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.816494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.817132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:57.817432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:57.818434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.818839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:57.830929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 317/626] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:57.831907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.866701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 310/627] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:57.868290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c03664-7aca-49b6-ac3a-9e0d5ca85715 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:57.868778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c03664-7aca-49b6-ac3a-9e0d5ca85715 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:57.869248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c03664-7aca-49b6-ac3a-9e0d5ca85715 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:57.870724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 311/628] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:57.871669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c03664-7aca-49b6-ac3a-9e0d5ca85715 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:57.961159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3ebd1-aa9c-442c-9ab3-e00029e0daf6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:57.969299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:57.990468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c03664-7aca-49b6-ac3a-9e0d5ca85715 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.004725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.005502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.006133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.006432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.027109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.028151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.028589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.029103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.036704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.038397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-11a6416b-9b55-4e4e-af66-fcfc8e111edb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:58.050224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:58.051274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:58.052127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 312/629] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:58.053048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.053141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.054264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.073830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:58.075167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:58.093439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.095023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.096855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.146346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 313/630] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:58.147536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.222534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 318/631] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:44:58.223420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.236108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.261508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d587-fffa-4e28-90fa-e70db4918cf2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.302839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.303928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.304429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.304888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.305793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:58.306549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.309253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.329028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 314/632] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:58.329028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.341419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a37a9c-fe6c-406d-aa93-d1a74550dbe8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.379067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.379572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.379847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.380119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.390898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:58.396497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:58.397414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:58.398215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:58.402631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 319/633] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:58.403607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.412324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.453883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7370c2-1b97-447b-ae3a-8e24f77f1366 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.476772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.491840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.492481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.492760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.493028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.503880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.505336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:58.520504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:58.521474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:58.527497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:58.527497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:58.527497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:58.527668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564eb58-f432-4a4e-b8b2-ea88ac1cd8c6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.527807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 315/634] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:58.527807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.564372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.565454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.565750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.566027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/09cae6567bc540bab88dc59c1ddee27b/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.582438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=09cae6567bc540bab88dc59c1ddee27b, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.583901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.597049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 320/635] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:44:58 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/09cae6567bc540bab88dc59c1ddee27b/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:44:58.597954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.630872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.643920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e591d-da77-4521-8ed9-cbb306c47408 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.682432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.683069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.683346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.683605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.684340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:58.684771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.686555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.697170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 316/636] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:58.698597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.723174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49ffce-ce07-4bfa-9c7c-4b19b222443e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.744459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.768961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.769730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.770678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.770678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.783940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-9a000fd7-bac2-4843-bad4-4c147a3a6aa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:58.789237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:58.790312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:58.791310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 321/637] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:58.792559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2a94-aee3-4049-89bc-c341c646ebe9 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.792559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2a94-aee3-4049-89bc-c341c646ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.792727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2a94-aee3-4049-89bc-c341c646ebe9 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.801324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 322/638] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:44:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:58.802609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2a94-aee3-4049-89bc-c341c646ebe9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.834256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a8220-92a4-4169-b247-a304cbb15fa2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.870426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:58.882214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.882879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.883151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.883289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.883942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:58.884237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.885619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:58.899786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 317/639] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:58.900635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:58.943048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2a94-aee3-4049-89bc-c341c646ebe9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:58.985871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:58.986680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:58.986822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:58.987091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:58.998104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:58.999619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-dd6702a4-1de3-46ba-890b-b169bcff01d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:59.015288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:59.016275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:59.017151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:59.017151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:59.017151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:59.017151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 323/640] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:59.017797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.017970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.018218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.035291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f20065-e836-4782-b28a-e74f3c526d2e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.042548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:59.043887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:59.069239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.077357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.078142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.078547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.078887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.079622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:59.079994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.081444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.094846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 318/641] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:59.094846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.135671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 324/642] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:44:59.135901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.155691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.213247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec70277-69c0-4d6d-8c65-30e8b692a27e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.251386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.252211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.252558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.252896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.261349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.263283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.270695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f396e23-7288-44ec-8f7c-8e9de99e2907 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:44:59.276399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:59.277586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:44:59.278769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:59.284404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 319/643] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:59.284404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.324489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.325589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.326072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.326588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.327439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:59.328520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.330564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.347238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 325/644] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:59.348474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.374163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.401190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11694c-cd3d-43c4-b1b2-63351f0587e0 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.410697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.436634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.437270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.437549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.437816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.448728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.450265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:44:59.465368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:59.466338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:44:59.467963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:44:59.467963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:44:59.467963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:59.471338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 320/645] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:59.472350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.475406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ab876-b983-4c97-8098-a1f62173f80d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.515446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.516737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.517032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.517308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.518047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:59.518880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.520745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.539773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 326/646] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:59.539773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.590330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530367c1-f586-439b-832d-51e27ec9e1e8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.627281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.627910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.628184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.628477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.633318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.636747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.638248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:44:59.650653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:59.651707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 321/647] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:59.652694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.667876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ee8b2-fd99-4761-98c4-4fae86e082c6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.678502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.706060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.706844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.707190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.707539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.718549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:44:59.723435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:44:59.726540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 327/648] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:44:59.727618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.776228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff290f0-2511-41b7-8a46-b0f5d7612640 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.814746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.815367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.815648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.815924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.816575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:59.816883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.818269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.831680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 322/649] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:59.832318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.839957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123df0d-ccee-4065-8eae-680203648c59 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:59.876047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:59.876716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:59.876986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:59.877255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:59.878002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:44:59.878307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:59.879829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:59.894400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 328/650] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:44:59.894400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:59.894400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.937754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:59.959562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309fe-319e-41da-9a3c-c6ad194d7b91 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.013783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.014451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.014734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.015010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.015772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:00.016096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.017583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.018036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4d2b0-204c-4b23-9441-2fde3e480c5e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.038638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 323/651] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:00.039659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.069956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.071709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.072261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.072737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.074146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:00.075094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.076787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.091124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 329/652] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:44:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:00.092445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.163817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1d8be-7780-4520-ae01-2c5af3a5bc17 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.168298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.205780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.206716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.207000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.207270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.212849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60881392-34b0-4389-9cbf-fcbfb938ea14 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.216425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.217891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.221435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:00.235979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:00.236948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 324/653] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:00.238558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.257479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.258113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.258418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.258701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.348171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.350313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.353114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9943-f7b6-4268-acca-539ecd418b1e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.393527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.394689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.395157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.395686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.407940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.409883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:00.426777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:00.427784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:00.428632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:00.428632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:00.428632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:00.435401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 325/654] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:00.436158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.438034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.478474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 330/655] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:00.479762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.503392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.570590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c51b-40fb-4160-be54-eabaf28e38bb None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.588342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b522d5-26d1-4334-a7a7-164020d43a2c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.612166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.612815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.613088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.613353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.614094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:00.614418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.616363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.633035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.634286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.634592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.635000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.635787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:00.636090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.637391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 326/656] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:00.638345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.642097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.658593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 331/657] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:00.659472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.697373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.774094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf515b57-7fad-41b4-909c-74b9714d1546 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.781431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c429f-9fab-46c8-8464-7d68c901fc5b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.786714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.816378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.817027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.817299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.817569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.826351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.827776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.828451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.828865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:00.828977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.830437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:00.846227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:00.847424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:00.848315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:00.848315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:00.848315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:00.848315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 327/658] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:00.848315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:00.912106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:00.913764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf9526-be84-44cf-8a2b-a849827e8b4f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:00.940030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:00.955957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c085-49e6-4652-b66e-6963eb0e84f9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:00.996239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:00.996948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:00.997229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:00.997543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.007014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.008579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-4e98ac3d-4dce-4382-a2c8-37a97100f745 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:01.021486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:01.022520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:01.023440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 328/659] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:01.024086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650a27-6f55-4e2f-ab08-e499a2e41668 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.024796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650a27-6f55-4e2f-ab08-e499a2e41668 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.024796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74650a27-6f55-4e2f-ab08-e499a2e41668 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.026064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 329/660] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:01.031319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74650a27-6f55-4e2f-ab08-e499a2e41668 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.040779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 332/661] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:01.040779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.040779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.040779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.056468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.060285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:01.061504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:01.131451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 333/662] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:01.132673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.153682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74650a27-6f55-4e2f-ab08-e499a2e41668 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.194383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.195021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.195268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.195540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.196191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:01.196491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.197938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.220654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.221000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 330/663] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:01.223624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.273179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf483e2-e3d9-47bc-8552-516c26cac7a0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.313722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.314269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.314579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.314852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.315594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:01.315919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.317368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.337390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.342558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 334/664] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:01.343543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.362758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df44aa-b50e-4eb0-b6c5-d2608e7d2d55 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.403153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.404214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.404736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.405194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.418043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.419984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:01.437634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:01.438681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:01.444441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:01.444441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:01.444441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:01.444441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 331/665] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:01.444441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.473173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8efd8-37ce-4eca-af0d-232246de1f71 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.488015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.510769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.511716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.512008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.512367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.519616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.521092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:01.533269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:01.534170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:01.535069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 335/666] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:01.535529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.566496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5a3c3-c0f2-4c16-abeb-deb5ea658336 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.603724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.609842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.610658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.610946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.611993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.646535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925994f7-b2b5-404d-922a-7981f5367dab None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.684978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.685643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.685997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.686194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.689167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.691038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.697719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:01.703229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:01.704213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:01.705036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 336/667] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:01.705710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.739947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.821549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76222d7c-4d28-4ef8-8f94-b778aad99211 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.821736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 332/668] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:01.821736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.852953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.862535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.863535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.864046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.865052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.866649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:01.867653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.870830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.895040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 337/669] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:01.895040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.933355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd6618-cb05-4be3-a6e3-dbf777453389 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:01.970644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:01.971235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.971566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.971842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.972552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:01.973505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:01.974316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:01.991575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 333/670] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:01.991575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.021628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.031614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb1bce-292e-446a-bcc7-bf5ca09435a6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.070811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.071496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.072473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.072473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.072652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:02.072892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.074273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.084033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 338/671] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:02.087549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.096436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.108727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834570-5edd-4145-ad6b-f677dbd927e9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.145212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.145863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.146145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.146448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.159812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.161087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:02.177633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:02.178692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:02.179664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:02.179664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:02.179664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:02.179664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 334/672] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:02.180315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.214021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fe0e-7fa0-4ea4-97d4-51c39d294342 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.251708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.252863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.253145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.253564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.260745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.262269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:02.274270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:02.275289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 339/673] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:02.281448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.296848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f216e70-0e81-4c39-b035-5a6e0ec2f9c4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.299912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.333844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.334399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.334684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.334950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.335702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=0d318e084073402e8d3385eb69c9685b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:02.336007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.337531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.344099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 335/674] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/role_assignments?user.id=0d318e084073402e8d3385eb69c9685b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:02.344962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.366714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.402623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc5f8-183a-445a-bb08-7db2611c5336 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.446654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.447320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.447595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.447863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.463221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.464757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:02.480813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:02.481822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:02.482721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:02.482721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:02.482721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:02.482721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 340/675] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:02.482952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.484967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910585e-ac1a-41ad-a3ca-9b1b932d4c00 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.546434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.546434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.546708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.546999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.548076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:02.549037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.550812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ef9db-754f-4af3-9715-86225d5a4fb4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.563601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 336/676] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:02.565223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04d62-d21f-4c6f-a3a5-747e877bc58f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.565223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04d62-d21f-4c6f-a3a5-747e877bc58f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.565693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04d62-d21f-4c6f-a3a5-747e877bc58f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.569653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 337/677] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:02.569653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04d62-d21f-4c6f-a3a5-747e877bc58f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.584000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.631413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb8324-c3bf-47ac-8729-b3657148fcaa None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.652364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.670029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.670694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.670978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.671256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.671958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:02.672265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.673725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.687074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 341/678] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:02.688337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.699953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04d62-d21f-4c6f-a3a5-747e877bc58f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.738099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.738783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.739026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.739301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.739961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:02.740253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.742011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a73549-9c77-4e96-b75b-79444f795a01 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.754930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 338/679] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:02.754930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.754930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.754930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.775775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:02.776986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:02.801084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7162-8192-43c1-805f-fa1e48fbab2e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.834734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.846592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.847551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.848025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.848307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.856195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.857676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.862775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 339/680] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:02.863589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:02.882690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:02.883670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:02.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:02.887227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 342/681] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:02.888231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:02.935536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.981044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8c2b5-2849-4626-a53d-3d52f08ab1a9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.012081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970ba5a-5c10-4545-aae8-5f206d8db697 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.019563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.020294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.020571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.020841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.021569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=1a83958d50934b24ae40da8760b2fb85) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:03.021873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.023477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.034405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 340/682] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/role_assignments?user.id=1a83958d50934b24ae40da8760b2fb85&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:03.034405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.076711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.078093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.078593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.079061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.098082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.099963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:03.123460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:03.124441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:03.125388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:03.128061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 343/683] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:03.133961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.142712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7657ec-c1e5-46c6-8c66-24a3bec7b41c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.170073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.184007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.184643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.184989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.185264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.186032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:03.186431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.188261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.205413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 341/684] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:03.206885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.221527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.297162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2aaf8-5299-48a3-9807-77a2bcf127d1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.338816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a79a7-99c0-44dd-ab93-b3bc9944b6bd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.339816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.340525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.341901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.342182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.344129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:03.344446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.345977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.372185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 344/685] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:03.373414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.396953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184abb68-6624-4406-8155-46cfea813880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.397614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184abb68-6624-4406-8155-46cfea813880 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.397892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184abb68-6624-4406-8155-46cfea813880 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.398161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184abb68-6624-4406-8155-46cfea813880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.413652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184abb68-6624-4406-8155-46cfea813880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-184abb68-6624-4406-8155-46cfea813880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:03.418779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:03.420083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 342/686] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:03.421349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184abb68-6624-4406-8155-46cfea813880 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.500153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99431087-1eb8-4677-b76d-7d705eb0e0de None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.503802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.525856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.543842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.544877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.545374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.545834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.546780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=f641d3de0a3a43ad8bc789483ffd5a72) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:03.547375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.549080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.562658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 345/687] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/role_assignments?user.id=f641d3de0a3a43ad8bc789483ffd5a72&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:03.562658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.563725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184abb68-6624-4406-8155-46cfea813880 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.625858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.626868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.627326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.627765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.752528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.756812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.762231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adaf57-b96c-47d9-99ee-69b378c55d0e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:03.825042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:03.825993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:03.827827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:03.828303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:03.829681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:03.830164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:03.832180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:03.833540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.845883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 346/688] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:03.852162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.886254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.906259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 343/689] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:03.912420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:03.968554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e50e8-f490-402d-894f-7ddccf569e50 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.008327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.009182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.009380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.009635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.025577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.027107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.038577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ccc4b-0cfc-47cb-a85d-e01592f7244f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:04.042994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:04.044224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:04.045084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:04.045084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:04.045084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:04.045084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 347/690] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:04.049976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.093275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.093937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.094215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.094520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.118361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.119883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.125232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:04.151967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:04.153040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:04.153978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:04.153978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:04.153978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:04.153978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 344/691] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:04.159955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.206972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:04.251270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef0782-7216-418c-af01-2f125d1b453a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.319450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5a004-7263-4697-81df-74385e1b9f23 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.356554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.357210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.357565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.357730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.358392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:04.358721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.360223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.363094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.363743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.364021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.364300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.364955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:04.365260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.367059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.374214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 345/692] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:04.387510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.393263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 348/693] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:04.398500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.504848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:04.588347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f547b5-f70f-4e75-946e-1baca5d273d9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.590091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0de3-0b58-4766-b0d6-7efb3e8e1c56 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.655761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.656521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.656807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.657208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.665501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.666998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.671423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.672079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.672356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.672625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.688104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.689849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:04.693105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:04.693995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:04.694888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 346/694] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:04.706982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:04.712698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:04.713858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:04.715141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:04.715765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 349/695] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:04.721670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.728711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:04.866136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba8d8-857d-4e12-9a21-6b788ff9bfdb None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.910188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f6875-c992-43b5-88af-0002f91ce99c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:04.912010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.914355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.914961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.915519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.916848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:04.917378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.919732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95813d99-2128-43da-ab81-b5efdb38f460 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.930088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:04.946848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 350/696] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:04.948371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab60c4-42fd-4817-8659-063587c4eb70 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.948666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab60c4-42fd-4817-8659-063587c4eb70 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.949020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab60c4-42fd-4817-8659-063587c4eb70 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.954414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 351/697] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:04.954414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab60c4-42fd-4817-8659-063587c4eb70 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:04.964627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:04.966147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.966821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.967402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.968599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:04.969133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:04.971594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b596b29c-5fa2-44ae-9074-3d078eeb30df None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:04.995179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 347/698] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:04.995179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.995179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.995179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.016508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:05.017860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:05.037462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.097918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab60c4-42fd-4817-8659-063587c4eb70 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.111266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 348/699] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:05.113266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.166968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.167517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.167791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.168388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.289128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.294747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.321581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0998ae5-36e3-4ec0-a568-245163f847e7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.326263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.385304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.391315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.391989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.392244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.392512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.393760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=09cae6567bc540bab88dc59c1ddee27b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:05.394078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.395740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.414075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 349/700] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/role_assignments?user.id=09cae6567bc540bab88dc59c1ddee27b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:05.436625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.459452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 352/701] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:05.468241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.668261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696790a2-3f72-4247-a41c-3a84ec7240e5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.669590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1393ad8-803a-4cd4-8353-b5b9dae5c2fd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.710314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.710914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.711194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.711463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.715329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.716189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.716687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.717103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.722873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.724376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.731058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.732744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a7911c10-82a1-4dc3-bab2-1e395d8f4f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:05.741945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:05.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:05.744028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:05.744028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:05.744028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:05.744028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 353/702] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:05.745206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29380-dbeb-41fc-a807-03db452a7db8 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.745206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29380-dbeb-41fc-a807-03db452a7db8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.745206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29380-dbeb-41fc-a807-03db452a7db8 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.749309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 354/703] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:05.749309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c29380-dbeb-41fc-a807-03db452a7db8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-0051af5a-9868-457b-a3a0-02788845ab74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:05.758847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:05.759968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:05.760823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:05.760823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:05.760823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:05.760823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 350/704] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:05.762278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.762278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.762278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.768164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.786027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:05.787313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:05.805364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.873384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 351/705] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:05.874414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.887744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c29380-dbeb-41fc-a807-03db452a7db8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.963200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.963899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.964180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.964455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.965178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:05.965484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.966925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.991906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 355/706] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:05.995881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.079744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb2caf-f7a0-4644-8c98-ee4d290fba12 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.128938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.147981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.147981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.147981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.148381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.149150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:06.149434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.155148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.168359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 352/707] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:06.169583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.172952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98b373-732f-438c-aa17-cd61447bfd27 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.184383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.219248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.219945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.220289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.220626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.230052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.232325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:06.248067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:06.249225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:06.250362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:06.251865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 356/708] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:06.253002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.303191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ff339-49cd-43a5-87a8-6f8a325f262e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.341379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.342571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.342861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.343136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] PATH_INFO: `/v3/services/network` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.343886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.345269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:06.356897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:06.358034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:06.359031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 353/709] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:06.359807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.382012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961baffe-9325-430c-930d-58aec32e5fd2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.414192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.419789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.420413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.421107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.422051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.422533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.424395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57f24b-d182-4def-8905-10d7eaccaba9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.446462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 357/710] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:45:06 2022] POST /identity/v3/users => generated 254 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1749b3-7034-4d7e-aa83-fa3042c6e728 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1749b3-7034-4d7e-aa83-fa3042c6e728 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1749b3-7034-4d7e-aa83-fa3042c6e728 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 358/711] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:06.462392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1749b3-7034-4d7e-aa83-fa3042c6e728 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.490338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efe9da-6846-4589-8f01-3e496119f86f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.546787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.547579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.548131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.548500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.549748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:06.550392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.551904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205fa6e-a0e8-4821-b3e2-d40131f79582 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.571072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 354/712] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:06.572506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.572797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.573072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.595811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:06.597187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:06.622173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1749b3-7034-4d7e-aa83-fa3042c6e728 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.677309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.678273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.678749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.679177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.680005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:06.680537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.681944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.682748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 355/713] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:06.683893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.693921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 359/714] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:06.695743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:06.759009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.762318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:06.806798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eb886-296f-43e7-99b2-d0534b7451ef None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.815673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec77c5-e5ae-42ad-9d42-7926a8b8b3f8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:06.846762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.847758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.848370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.848930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.849946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:06.850582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.851158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:06.851995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:06.852280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:06.852550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:06.853183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] RBAC: Adding query filter params (type=network) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:06.853365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.853473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:06.854865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:06.860542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 356/715] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:06.865000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 360/716] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:06.968016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:07.105828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79f71-0c38-4c71-9451-e7d8f4220ff7 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:07.137040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:07.147709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:07.148466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:07.148787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:07.149063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:07.149754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:07.151185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:07.161354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:07.162411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:07.163244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:07.166497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 357/717] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Sep 13 19:45:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:07.169413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:07.296645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0bce-7ff4-419e-8188-6d817fc30704 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:07.321367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:07.344393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:07.345026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:07.345360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:07.345639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:07.346296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:07.346632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:07.348029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2acb8-5e5d-4482-9496-4933d05041a6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:07.354542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 361/718] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:07.360845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:07.507128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a79fa-f0eb-4993-9e76-745283560fa8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:07.562506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:07.563149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:07.563435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:07.563705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:07.564358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:07.564662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:07.568714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0adb1-436b-416f-ae8f-8d7e095ec35b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:07.576003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 358/719] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:08.021492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f2d5-35ac-4460-a321-b2986c070058 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:08.022042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f2d5-35ac-4460-a321-b2986c070058 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:08.022417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184f2d5-35ac-4460-a321-b2986c070058 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:08.024039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 362/720] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:08.029840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:08.030127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:08.030419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:08.052588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:08.053786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:08.113711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:08.143292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 359/721] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:08.485685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184f2d5-35ac-4460-a321-b2986c070058 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:08.647058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184f2d5-35ac-4460-a321-b2986c070058 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:08.696465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:08.697506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:08.698013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:08.698520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:08.715432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:08.720773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:08.721740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:08.722584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:08.725651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 363/722] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:08.731867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:08.849137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7710f58-e489-4369-8fd9-f374b47c102b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:08.890313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:08.891366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:08.891872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:08.892340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:08.893222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:08.893725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:08.895379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:08.905810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 360/723] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:08.912181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:08.967952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:09.055126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b716f5-55a4-4912-aae5-664c34beeb4b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:09.107241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:09.107826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.108112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.108383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.119803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:09.121331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:09.141802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:09.142907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:09.144158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:09.144158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:09.144158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:09.144158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 364/724] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:09.148754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:09.236077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:09.298416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e9b2b-6aca-4d24-9af7-fc9dd81af203 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:09.335809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:09.336433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.336712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.337045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.337701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:09.338056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:09.339502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fea72d-1371-4f43-965f-b4d4694ffe2b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:09.352546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 361/725] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:09.358452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:09.410320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423967-2548-4871-b131-a61fd9454f0d None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.410994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423967-2548-4871-b131-a61fd9454f0d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.411693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423967-2548-4871-b131-a61fd9454f0d None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.414092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 362/726] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:09.424186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e241ea-83d4-46cd-8742-1ce04e91b834 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.424885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e241ea-83d4-46cd-8742-1ce04e91b834 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.425357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e241ea-83d4-46cd-8742-1ce04e91b834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.459338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-42e241ea-83d4-46cd-8742-1ce04e91b834 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:09.463209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-42e241ea-83d4-46cd-8742-1ce04e91b834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:09.583424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 363/727] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:45:09.583645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:09.591466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46427c2-d54d-4a9e-8dc6-c61054f7927d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:09.635834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acf4dc-bca2-4d05-932c-4e889a9bf784 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.636017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acf4dc-bca2-4d05-932c-4e889a9bf784 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.636306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acf4dc-bca2-4d05-932c-4e889a9bf784 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.637980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 364/728] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:09.643404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1d38-da3f-459e-b686-536f543f1811 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.643700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1d38-da3f-459e-b686-536f543f1811 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.643964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e1d38-da3f-459e-b686-536f543f1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.648420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:09.649368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.649850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.650582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.666312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1f7e1d38-da3f-459e-b686-536f543f1811 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:09.667545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1f7e1d38-da3f-459e-b686-536f543f1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:09.670294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:09.671915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:09.698180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:09.699295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:09.700227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 365/729] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:09.708627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:09.741098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:09.812881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 365/730] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:09.838513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9210-0731-4386-9118-4513e065b6e9 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.838816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9210-0731-4386-9118-4513e065b6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.839068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf9210-0731-4386-9118-4513e065b6e9 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.841157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 366/731] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:09.852341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.852838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.853184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.877935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12e72a-a1dc-4cd3-a3b9-e05286d35b06 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:09.886484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:09.887962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:09.916059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:09.917139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:09.917598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:09.918408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:09.919760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:09.920252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:09.922177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:09.947850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 366/732] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:45:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:09.953862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:09.966342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:09.996328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 367/733] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:45:10.003461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.108276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de788510-2aab-4617-acdb-8026d3ea228b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.175036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.175873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.176156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.176445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.180425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.185303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412facb4-f3f3-4c10-8e0a-bf2b8e4d2229 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.193357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:10.199892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:10.200948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:10.201762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 367/734] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:10.203067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.233101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.233723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.234524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.234816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.294274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.318423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.319895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.332642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1126975-a6ca-4f3d-8dd7-bbfdbd6a6337 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.370029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.370706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.371131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.371407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.383095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:10.388249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:10.389202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 368/735] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:10.391325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.432687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.445913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 368/736] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:45:10.446942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.510694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1010c6-9700-4d8a-bfca-0ccd72e12b89 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.545709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.552894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.553523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.553799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.554075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.554788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:10.555088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.556482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.561092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02c5f-add0-46be-ac76-1857328a3fee None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.566699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 369/737] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:10.569614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.598654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.599279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.599560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.599838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.600501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.601883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.620416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 369/738] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:10 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:10.621293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.678664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4ca9-3ffc-4167-8fd4-fd6dd1dfa38a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.696029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.728849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.729520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.729795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.730066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.730891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:10.731191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.732609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a50861-5b76-40b1-adc0-31fbc9e79f98 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.743277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 370/739] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:10.744600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9901b-0eb2-47f0-ac27-4ed2e49f4556 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.744891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9901b-0eb2-47f0-ac27-4ed2e49f4556 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.745164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9901b-0eb2-47f0-ac27-4ed2e49f4556 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.746656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 371/740] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:10.751447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9901b-0eb2-47f0-ac27-4ed2e49f4556 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.758069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5c42a-75f5-453a-b50b-0d40750bf14f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.797473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.798267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.798599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.798873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.811077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.812557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.814045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:10.833060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:10.834077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:10.834979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:10.834979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:10.834979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:10.834979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 370/741] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:10.835210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:10.879587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9901b-0eb2-47f0-ac27-4ed2e49f4556 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.915843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.916689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.917153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.917421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.929882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:10.931557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:10.947113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbd3c7-7288-4c98-94b6-82b7e98d1f25 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-07de5a3b-14e4-4f67-9f18-ecdfd6c791a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:10.949043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:10.949929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:10.951232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:10.951232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:10.951232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:10.951232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 372/742] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:10.951711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.951994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.952262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:10.964139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:10.972370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:10.973575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:10.994540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:10.995185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:10.995459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:10.995719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.007473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.008953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:11.025129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:11.026115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:11.032130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:11.032130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:11.032130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:11.032130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 371/743] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:11.032130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.051360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 373/744] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:10 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:45:11.052453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.130140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.163532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717ad46-370f-420a-b9b6-27a3083175be None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.203020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.203573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.203838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.204186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.204828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:11.205135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.206564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.226939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 372/745] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:11.228180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.256407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815cb5d-fe5b-42a3-9028-b3706b596cb6 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.259242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.301226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.302270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.302712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.303127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.304004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:11.304455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.306135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.319180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 374/746] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:11.320597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.339549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b70a8f-18f2-43ee-b82d-35cf7bf98663 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.375861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.376488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.376767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.377040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.377698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:11.378011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.379411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.393622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 373/747] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:11.394872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.421101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.432962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0830-8ce4-4894-9a05-e9090e238d39 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.468822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.469473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.469792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.470166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.477173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.478654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.488586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:11.490855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:11.492105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 375/748] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:11.493219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.515565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb7c6-b1b4-4765-88cd-895b1419476e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.561032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.561641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.561915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.562185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.570933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.572396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:11.587464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:11.588789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 374/749] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:11.595045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.624259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb61dc-fead-44be-918e-38ddeb27e916 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.667165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.668204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.668686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.669256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.670292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.672059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.692862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 376/750] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:11 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:11.694035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.712142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.728774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484d10a-7e6f-446d-a9da-21e749d7b530 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.791174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.792644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.793158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.793669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.802566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.805578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.808120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:11.822667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:11.823731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:11.824726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:11.827910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 375/751] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:11.829104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.834785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2205-8a1c-41e7-9849-c4348792f12f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.894978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.894978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.894978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.894978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.895898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:11.895898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.897391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:11.910711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 377/752] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:11.915346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.990780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78985832-a125-4e6f-b059-2c078fb651cc None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.028504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.034551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.036618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.037136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.038215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.039607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:12.040148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.042307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.066787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 376/753] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:12.070431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e0ab9-6dea-48aa-b097-8592a314c8b7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.074202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.118328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.119689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.120179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.121579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.122708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:12.123200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.125001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.155436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 378/754] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:12.156074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.156160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.212031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd0313-f45a-416d-9ce1-812b47168797 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.256365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.257914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.258636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.259075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.260111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=09cae6567bc540bab88dc59c1ddee27b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:12.260570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.262392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.270910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 377/755] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/role_assignments?user.id=09cae6567bc540bab88dc59c1ddee27b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:12.278645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.298304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6142e6-0219-4973-934f-67eb29b717f2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.370774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.374499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.375194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.375459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.375753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.394083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.395629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:12.438221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:12.439324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:12.440202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:12.440202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:12.440202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:12.442494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 379/756] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:12.458697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.461013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e5adb-5d6c-4b77-b931-6caba4c9b9f9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.483701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.516336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.516889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.517169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.517439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.615713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80deec36-0af6-4a12-bd6e-8e773986c551 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.661155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.662175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.663166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.663653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.664546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:12.665045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.666663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.681715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 380/757] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:12.687920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.701883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.703775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f3fd3-0bea-4c2e-b7ce-e5fb44e8708c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.796327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.876444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 378/758] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:12.878599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c77-59b8-4f73-a522-ccb644f65fe2 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.879187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c77-59b8-4f73-a522-ccb644f65fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.879636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c89c77-59b8-4f73-a522-ccb644f65fe2 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.882974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 379/759] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:12.884004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89c77-59b8-4f73-a522-ccb644f65fe2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:12.896485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe3f9b-e66a-4792-89ca-9b740db35e54 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:12.909767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.934322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:12.935240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.935657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.936125 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.943764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:12.945410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-cc278a3d-b8ae-4970-9521-ca57eb9b0c92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:12.957712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:12.958854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:12.959887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:12.961991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 381/760] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:12.963459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.963885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.964310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.985696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:12.987247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:13.010066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c89c77-59b8-4f73-a522-ccb644f65fe2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.053614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.054289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.054646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.054980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.066496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 382/761] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:13.066747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.068038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.068525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:13.086429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:13.087604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:13.088502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:13.088502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:13.088502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:13.093806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 380/762] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:13.095564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.167789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.175132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.210939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1f37c-dd75-40b5-ae09-bb3ffde2313a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.249226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.249847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.250126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.250411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.251067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:13.252089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.252880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.269637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 381/763] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:13.271902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.319782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02786ec-1ae8-44cb-ad90-13ac1b562cb0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.382174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9688-8305-4e62-b570-66128e65b268 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.417525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.418206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.418518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.418892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.424441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.425089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.425373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.425747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.426760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.427767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:13.428078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.428385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.429940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.430135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:13.443701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:13.444641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:13.445512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 382/764] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:13.446603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.454895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 383/765] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:13.457185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.570125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46f5e6-e9f1-4efe-9541-c862c41eccec None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.608690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.609730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.610781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.611264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] PATH_INFO: `/v3/services/network` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.612059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.614486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:13.626441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:13.627453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:13.628271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:13.636280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 383/766] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:13.636280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.707712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.720636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.734834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9526a-aae1-482c-ad61-c81c36462f30 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.763205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108eaf5e-9da1-40b9-ac56-6092fd6dcb93 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.783779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.784412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.784691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.784957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.785657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:13.785977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.788113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.806464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 384/767] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:13.807732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.809878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.810601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.810882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.811173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.813214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=463fcb701698473bad12e8101b9018ce, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.815880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:13.833502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 384/768] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:13 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:13.838941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.028635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8b68-9b9a-4aaf-b484-9a84957a069d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.069992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6e0e4-db79-4e95-8f44-6b58af3d5cfd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.081447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.082649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.083223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.083740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.084643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] RBAC: Adding query filter params (name=network) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:14.085132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.086832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.090703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:14.093639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 385/769] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:14.102994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.118811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.119458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.119736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.120434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.121534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=398ef52b1f264cab85425daaa03e2b6a) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:14.121984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.123693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4692-c293-4e13-9ebe-76d27dcfdfdf None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.127932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:14.138397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 385/770] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/role_assignments?user.id=398ef52b1f264cab85425daaa03e2b6a&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:14.162773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a74e-05c5-4d43-be87-69938398cecf None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.163090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a74e-05c5-4d43-be87-69938398cecf None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.163323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450a74e-05c5-4d43-be87-69938398cecf None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.165624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 386/771] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:14.172574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.172990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.173437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.195954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:14.197393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:14.285748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6484e34-62f2-45d9-aa2b-cae545fee6cd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.298738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 387/772] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:14.339729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.340273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.340555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.340827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.341486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:14.341789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.343274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.350108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 386/773] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:14.356502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.472274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa07d4-f066-4b53-8f67-caba8df9b051 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.484337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:14.509475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.510293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.510622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.510898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.511699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] RBAC: Adding query filter params (interface=public, service_id=81d9f94daba74a2f8a2c2c2f80eca543, region_id=RegionOne) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:14.512008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.513394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.522484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 388/774] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/endpoints?service_id=81d9f94daba74a2f8a2c2c2f80eca543&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:14.528687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.618519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:14.641618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e85d8f-9ad8-4d4b-b194-19b0883902d2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.683615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.684242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.684520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.684786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.685391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.687204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee18b4d-9ddb-4122-94cb-10fadfa12db6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.692630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.696590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 387/775] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:14.782265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af9593-2b48-435d-a084-a87f4930191c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.782869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af9593-2b48-435d-a084-a87f4930191c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.783349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af9593-2b48-435d-a084-a87f4930191c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.784896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 388/776] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:14.791600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.792163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.792705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.836145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:14.838683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:14.869382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46115969-eb9c-42ed-ba21-013ce5a28038 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:14.930860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:14.931687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.932151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.932549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.933360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:14.934863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:14.944932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:14.945970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:14.946841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 389/777] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:14.955396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.972222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 389/778] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:15.014296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.083761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816706e7-f72e-4b1e-b9e0-239c47c53d5e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.126146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.126692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.126973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.127251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.128102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:15.128424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.129867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:15.135337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 390/779] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:15.141229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.272583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.289388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.305084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cbf55-fba9-4e4b-bf42-7297000a4240 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.346110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.346792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.347074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.347352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.348683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:15.348996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.350531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:15.357260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 390/780] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:15.363535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.515347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1299c-99c0-4cd0-9ce9-86708a8cc63a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.518694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.588653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.589826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.590258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.590701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.610123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:15.612345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f2b87-8ebe-456c-b400-0b441b632d4e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:15.620934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:15.621940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:15.622803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:15.630521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 391/781] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:15.633527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.687275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.688425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.688909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.689469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.690747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] RBAC: Adding query filter params (interface=public, service_id=60992e8660fe4f70a12c7aec72faa3e6, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:15.691607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.693645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:15.703115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 391/782] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/endpoints?service_id=60992e8660fe4f70a12c7aec72faa3e6&interface=public®ion_id=RegionOne => generated 194 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:15.709447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.712880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.838670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5950bd-9722-42b7-9e73-c90a0db22e75 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.906785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.907867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.908491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.909878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.911042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:15.912163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.913956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:15.916057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc11c58-fa06-48f2-a092-8749105effda None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.933646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 392/783] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:15.939526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:15.968410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.991251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.992312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.992791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.993273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.994096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.995773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.009311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 392/784] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/services => generated 772 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:16.080664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6c19-296d-460e-a2b4-bf4a25899575 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:16.130786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:16.131452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.131699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.131892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.149889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:16.151462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:16.172234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:16.173238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:16.174127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:16.174127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:16.174127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:16.174127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 393/785] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:16.179535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.274101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:16.317522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a755301-b674-42f3-8bbe-81d1c834998d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:16.353777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:16.354808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.355358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.355861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.356706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:16.357207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:16.358822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7916cf-3cf4-42da-abf9-ed7695e5dfa4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.372598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 393/786] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:16.377888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.402784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f73a-84d6-453c-95d4-b9e9966217b6 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.403378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f73a-84d6-453c-95d4-b9e9966217b6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.403846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f73a-84d6-453c-95d4-b9e9966217b6 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.405657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 394/787] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:16.411428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.411995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.412505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.432751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:16.434350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:16.496221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:16.582634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940dadc-f99d-4d98-a119-8e0c655cfe31 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:16.624673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 395/788] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:16.637278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:16.638209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.638711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.639147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.648203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:16.649985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:16.662578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:16.663562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:16.664481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:16.666759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 394/789] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:16.675605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.754487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:16.805005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89148dd-b1a4-4041-bcd8-7e18a12a29b9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:16.841423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:16.842333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.842830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.843242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.844225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:16.844666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:16.846247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.865890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 396/790] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:16.871448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.983566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.995729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfe502-4dc1-42f6-9e1d-cf3af40aac5d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.054794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.055440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.055712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.055977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.066512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.071838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.073469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:17.098789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:17.099787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:17.101258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:17.101258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:17.101258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:17.101258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 395/791] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:17.107887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.146207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ada3c-8a5c-4923-baaa-9170f133b4d6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.161867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.186070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.187142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.187738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.188017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.201948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-711edc4c-83c3-443c-bcc8-e4eb731fc70a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:17.207130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:17.208096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:17.208902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:17.213706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 397/792] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:17.215958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438dc34-c537-4c7e-9bef-aaa05a49ef7d None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.216245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438dc34-c537-4c7e-9bef-aaa05a49ef7d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.216521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438dc34-c537-4c7e-9bef-aaa05a49ef7d None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.222529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 398/793] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:17.223613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438dc34-c537-4c7e-9bef-aaa05a49ef7d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.351925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438dc34-c537-4c7e-9bef-aaa05a49ef7d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.365702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91395f76-88ab-4339-bb60-21ef2ee119be None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.406272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.407432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.407857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.408265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.408376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.409083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.409408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:17.409517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.409876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.410079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.410734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:17.411131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.411947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.412689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea764174-f05b-4b0a-ba9b-6677d1481aca None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.427993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 396/794] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:17.427993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.427993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.427993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.429042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 399/795] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:17.430273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.448106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:17.449530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:17.474911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.504346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.537692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 397/796] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:17.538918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.553914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b98b3f-05b8-4b44-be67-fb5e24259080 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.599638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.600296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.600580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.600861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.612995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.614489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:17.631601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:17.632587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:17.633479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:17.633479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:17.633479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:17.633479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 400/797] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:17.653501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.693778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0604af2a-a47a-499c-ad51-5df6755881fa None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.728496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.729216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.729492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.729755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.736956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.738518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-26d3723b-cdc0-4100-9794-020d46e8a135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:17.750791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:17.753002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 398/798] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022be0f-9063-422d-addd-a526ddba9d0f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.755192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022be0f-9063-422d-addd-a526ddba9d0f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.759418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022be0f-9063-422d-addd-a526ddba9d0f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.759418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 399/799] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:17.762481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.763189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.763461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.763722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.778545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a62e20-6c6a-4a1a-8a3c-6f422f7624ba None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:17.785083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:17.786397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:17.816119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:17.816847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.817133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.817409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.818157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:17.818506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:17.819999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:17.820624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:17.835529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 401/800] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:17.836644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.870613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 400/801] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:17.872327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:17.971842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e86618-c6e4-4755-b2c6-19d195c71b06 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.016966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b208ea-2743-4003-a2e3-08a635a663d7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.020713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.020713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.020983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.021150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.021846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:18.022154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.023652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.036797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 402/802] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:18.037569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.049766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.063481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.064141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.064419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.064746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.072494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.074118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:18.087080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:18.088158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:18.089164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:18.093003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 401/803] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:45:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:18.093902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.132272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.157971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704639d-aa06-4ced-8c64-bf8c48267cfb None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.197890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.198736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.199022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.199290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.210149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:18.215516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:18.216437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:18.217254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 403/804] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:18.217958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.220135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af65fd-9054-4e65-9be8-be302412548f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.260198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.260994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.261341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.261674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/09cae6567bc540bab88dc59c1ddee27b/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.263906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=09cae6567bc540bab88dc59c1ddee27b, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.265500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.282431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 402/805] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:18 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/09cae6567bc540bab88dc59c1ddee27b/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:18.283402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.304479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.348921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29392f-a961-4101-8d59-0f97608305c4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.391103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.391755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.392026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.392184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.392909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:18.393214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.394806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.419504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.421685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 404/806] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:18.422971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.440776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61ecfe-8885-4eab-a677-9c07df669d72 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.488131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.489229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.489710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.490842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] PATH_INFO: `/v3/services/network` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.491464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.492924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:18.506066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:18.507378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:45:18.508208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:18.512138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 403/807] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:18.513325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.554652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a34f36-3c6d-44a0-a9d9-81e51c799d46 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.586910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.599520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.600804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.601514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.602105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.603313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:18.603871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.605596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.617990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 405/808] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:18.619513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.659278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029117fc-8ab3-48ca-8e17-72282333c538 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.705816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.706607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.707374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.707648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.736161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.737974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74630cee-5738-4a95-ba46-4c99999ed801 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.782951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.783934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.784315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.784746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.785560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] RBAC: Adding query filter params (name=network) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:18.785923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.787719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.793824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 406/809] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:18.794581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.799725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.801187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cea1b-5a13-45fe-b9c3-52b209f8f193 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.858961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.918062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98125d6c-376a-40cf-bbe4-c7cfb2400c16 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.920952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 404/810] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:18.922262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b02d8-dd76-47b4-ba00-da89524af47e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.922576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b02d8-dd76-47b4-ba00-da89524af47e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.922831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b02d8-dd76-47b4-ba00-da89524af47e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.924416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 405/811] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:18.929646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b02d8-dd76-47b4-ba00-da89524af47e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.971381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.972056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.972338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.972749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.985971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.988127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:19.004605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:19.005642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:19.006570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:19.006570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:19.006570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:19.010839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 407/812] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:19.011809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.012323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.076478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b02d8-dd76-47b4-ba00-da89524af47e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.119286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.120033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.120307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.120773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.121433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] RBAC: Adding query filter params (type=network) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:19.121731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.123178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf2e81-6074-401c-8a47-f4f768e16917 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.127063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178ac05-d2df-449d-a4bb-e1cb61f972d0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.134330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 406/813] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:19.134545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.134626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.134626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.134626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.151061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:19.152338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:19.168403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.169004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.169281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.169548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.180962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.182462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:19.196866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:19.198023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:19.198992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:19.198992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:19.198992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:19.198992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 408/814] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:19.198992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.232890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 407/815] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:19.234439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.303211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.321657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd76e4-f7c1-4d6c-9690-54687fa99020 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.356611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.357230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.357471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7394a13-6fd8-46e9-a225-2bc2a3789131 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.357578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.357780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.358426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:19.358730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.360121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.373117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 409/816] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:19.373972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.394673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.403844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.404607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.405048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.405321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.406329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.407991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.438917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 408/817] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:19.439781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.530853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc8946-df18-43ec-9866-aa722d0ac289 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.590898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.597495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.598479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78f1c-60ab-4ea3-8130-f46082cc95b1 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.598860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.599348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.599812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.600681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:19.601387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.604020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.620831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 410/818] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:19.622261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.645943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.646732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.646926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.647201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.656635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.658148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:19.671496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:19.672478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:19.673376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:19.677535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 409/819] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:19.678562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.716311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.755075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c635-a1bc-4b02-98ee-98c476777cfc None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.795901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.796256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3842d0-5623-48c0-bf65-6329b8651518 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.796548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.796827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.797099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.797694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.799109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:19.811339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:19.812425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:19.813911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 411/820] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:19.814760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.839029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.839740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.840210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.840489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.849198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.851443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:19.862522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:19.863451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:19.864324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:19.866516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 410/821] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:19.867337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.888329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.926989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d82871-2794-4066-bf36-10114ffa9a90 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.962442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.963058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.963334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.963604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.964427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:19.964732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.966273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.969188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.977276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efda42b-80bf-4f14-8286-0e0170b52801 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.991104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 412/822] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:19.991843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.019373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.020017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.020268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.020529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.021166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.021467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.022888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.032609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 411/823] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:20.032609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.104287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c5c15-30b7-4df3-b5b1-a3cfce955b7c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.119981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.140712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c13ff-980f-40a1-96ac-b4a39f5d6727 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.140909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.141334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.141612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.141885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.142669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.142971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.144428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.156993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 413/824] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:20.157830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.186484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.187137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.187401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.187664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.200116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.201579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.217572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:20.218599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:20.219941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:20.225935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:20.225935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:20.225935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:20.225935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 412/825] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:20.225935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.271831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f98a4-9df8-4749-baf3-b6069284382c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.308207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.308830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.309109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.309381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.310020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.310638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.311735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.317368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 414/826] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:20.318066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.343576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f1f58-e94d-413a-b41f-6f18c897546d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.379663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.380387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.380664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.381072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.381826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.382103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.383614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.385751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.398587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 413/827] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:20.400200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.426684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67895d4-e6b5-4b8d-bbde-b3f99cc26f90 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.451947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.473554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.474141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.474453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.474729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.475402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.475713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.477156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.495705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 415/828] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:20.497277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.575597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c445d1-8732-4d37-ab65-487353c8d47f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.614053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.614711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.615925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.615925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.616286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.617962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21058a5-c5dc-44dc-9fff-a1765575dd1c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.625904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498e97-e810-4c2f-8c69-04a4e5804e75 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.640386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 414/829] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:20.641749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881acb7-5333-43b6-95b5-8628393dcdb3 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.642023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881acb7-5333-43b6-95b5-8628393dcdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.642289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881acb7-5333-43b6-95b5-8628393dcdb3 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.643817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 415/830] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:20.650338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.650710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.650909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.678164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:20.679748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:20.681685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.682552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.683026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.683407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.693975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.695510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:20.709541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:20.710782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:20.712152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 416/831] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:20.717394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:20.750334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.772096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:20.772435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 416/832] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:20.851752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ce159-44bd-4b0b-a75e-e3d4400ef10c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:20.892119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:20.892758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:20.893036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:20.893305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:20.893996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:20.894297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:20.895778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:20.908859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 417/833] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:20.915167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.027838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279ee80-1c74-4003-85cc-2e8c7e122f3c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.040809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.076057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.077107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.077577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.078294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/398ef52b1f264cab85425daaa03e2b6a/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.098479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=398ef52b1f264cab85425daaa03e2b6a, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:21.100791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:21.127063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 417/834] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:20 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/398ef52b1f264cab85425daaa03e2b6a/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:45:21.154787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:21.159093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639214e-a123-4c25-9d25-fc30e1ea1eac None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.199748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.200422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.200701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.200972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.213881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:21.219716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:21.220748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:21.221984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 418/835] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:21.227711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.268508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:21.352537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4c01d-8824-49a3-a49f-36b051590447 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.395991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.396894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.397309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.397738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.398621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:21.399071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:21.400626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:21.411190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 418/836] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:21.416874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.524297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c7909-91cc-4fe1-bb0b-7500a0219a48 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.560886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.561862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.562318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.562773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.578396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:21.578396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:21.591779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:21.592759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:21.593619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:21.593619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:21.593619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:21.593619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 419/837] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:21.598852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.698000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:21.705092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c47e9a-49da-4c4e-a294-e7e722e2a80c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.744096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.745168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.745662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.745939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.746624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:21.746924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:21.748352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:21.765754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 419/838] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:21.771192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.782750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:21.886397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff0346-22b3-4c62-b407-d7e57960381b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.921260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.921908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.922185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.922494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.930726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:21.932201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:21.944085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:21.946251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:45:21.948600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:21.950428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 420/839] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:21.954399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.054218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d06f31-c468-49d5-ad12-0b13f1b8d054 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:22.089061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:22.089686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.089960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.090226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.090984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:22.091280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.092704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:22.111770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 420/840] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:22.119706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.206754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.247285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682a6f6-c4b7-40aa-a73a-31cf77e0a390 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:22.288265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:22.288885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.289160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.289427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.362751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.364287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:22.443328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.471203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 421/841] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:22 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:45:22.479516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.499716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.610612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e74a5-687d-4a89-b4c6-59a6e949e09c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:22.655836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:22.656948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.657477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.657974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.670946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.672880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-bdb24c50-6d64-4c68-a739-74b30fa9055c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:22.689933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:22.690990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:22.691871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:22.691871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:22.691871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:22.694239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 421/842] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:22.700572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.761123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23605885-c5e9-48d3-89b6-46c19433876e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.761323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23605885-c5e9-48d3-89b6-46c19433876e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.761583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23605885-c5e9-48d3-89b6-46c19433876e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.763262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 422/843] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:22.768821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.769103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.769378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.789098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:22.790607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:22.815509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9e93-9669-465c-b0d9-eb831d50a2ea None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:22.857519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:22.858157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.858496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.858771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.859425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:22.859726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.861126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:22.874422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 422/844] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:22.880176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.888780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.900894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 423/845] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:22.916039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:23.012035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0f99e-4791-4179-a68c-c84b855d4c14 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.058273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.058969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.059251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.059526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.067446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.068933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-57171d73-f420-440f-8ced-bab20dc67792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:23.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:23.082717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.083629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 423/846] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:23.090800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.111005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3977be-c72a-4f3b-803b-cdcc95713423 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.111331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3977be-c72a-4f3b-803b-cdcc95713423 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.111561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3977be-c72a-4f3b-803b-cdcc95713423 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.113263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 424/847] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:23.119610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.119930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.120199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.140470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:23.142022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:23.206490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8713-d09e-4ee9-8ead-da8fdb415fc3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.245719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 425/848] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:23.247604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.253586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.254228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.254554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.254822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.255527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:23.255824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.257284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.278063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:23.279146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 424/849] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:23.280594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.341408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:23.382811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab8f3-302b-4460-b19e-0c47af776f29 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.391178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc927-804d-4e06-af2c-66bd18addc51 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.424228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.425536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.426068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.426633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.427598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.428459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.429152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.429495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.429834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.430141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.430784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=09cae6567bc540bab88dc59c1ddee27b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:23.431270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.432946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:23.441472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:23.442516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:23.443432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.450466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 425/850] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/role_assignments?user.id=09cae6567bc540bab88dc59c1ddee27b&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:23.451298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 426/851] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:23.455603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.529690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.634691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93241b-4930-45a0-8266-5c54755221da None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.658873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:23.673976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:23.691990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788b43-93ae-4e74-b2fe-d56ab682b6b9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.700474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.701108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.701379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.701642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.702293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:23.702654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.704056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b20cf-6499-43a3-b777-149af47204d5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.710210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 426/852] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:23.711887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891407d-cae2-46c8-8cb5-9307bfbdacf7 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.712166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891407d-cae2-46c8-8cb5-9307bfbdacf7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.712431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891407d-cae2-46c8-8cb5-9307bfbdacf7 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.713719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 427/853] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:23.717683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891407d-cae2-46c8-8cb5-9307bfbdacf7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.751983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.752739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.752906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.753152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.753752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.755232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-69063178-bbf6-4bee-9290-b45895e26d3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:23.765430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:23.766579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:23.767903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 427/854] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:23.768696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.768696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.768814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.789092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:23.789570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:23.859243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891407d-cae2-46c8-8cb5-9307bfbdacf7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:23.883307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 428/855] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:23.884619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:23.905621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:23.906674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:23.907143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:23.907614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:23.908440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:23.908940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:23.910778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.916672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 428/856] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:23.998761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:24.034078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2eff4-3940-43c7-adda-ca2d3c2cfe4c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.082242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.082849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.083137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.083411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.084143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:24.084454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:24.085895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:24.092046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 429/857] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:24.098041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:24.164631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:24.182317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:24.249735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3950015-931c-46e6-bfa3-318d2f6a2300 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.295146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.295790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.296068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.296340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.297002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:24.297313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:24.298749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:24.310399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 429/858] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:24.310612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5ce6-9521-4745-8919-67f3b7099d04 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.355231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.355865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.356148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.356477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.369634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:24.377074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:24.383945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:24.384968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:45:24.385851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:24.392214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 430/859] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:24.398154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:24.520490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b7d06-d407-4062-9930-883e6b52bfb8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.550907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:24.577566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.577566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.578054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.578232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.578946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:24.579249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:24.580694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:24.591470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 430/860] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:24.597672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:24.732088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17bdec-c3dd-49ae-bb6b-90ef4cd3dfe6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.772901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.773704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.773937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.774215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.787334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:24.788892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:24.805770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:24.806823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:24.807983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:24.807983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:24.807983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:24.807983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 431/861] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:24.813419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:24.936216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10654-9d24-4af3-bc33-d523d177c792 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:24.975766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:24.976472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:24.976744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:24.977020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:24.977689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:24.977993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:24.979472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:24.998616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 431/862] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:25.000454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.057203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.112201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b945b-5fe6-46e7-b9f5-2fe0e48c6cf4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:25.150984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:25.151725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.152057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.152337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.161151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:25.162684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:25.175519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:25.176493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:25.177411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 432/863] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:25.183000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.222698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.314499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0b08b-e022-4248-b5a8-171b2a5985e3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:25.352714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:25.353427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.353721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.353989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.354854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:25.355155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:25.357220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:25.379967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 432/864] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:25.385886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.503862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f890f-fb43-4e9d-9c06-ba7425481783 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:25.515717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.541132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:25.542254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.542760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.543242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.622697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:25.624574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4f5c8-609d-4538-8745-8f3da5d1443a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:25.737470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 433/865] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:25.743470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.843482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.852897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432242-5a7b-4f72-9041-1f2c1a54d59a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:25.860050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ac38-8160-44d6-80f2-ced6f8c8fe0a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.860566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ac38-8160-44d6-80f2-ced6f8c8fe0a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.860977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82ac38-8160-44d6-80f2-ced6f8c8fe0a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.863635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 434/866] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:25.872865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.873292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.873732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.892610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:25.893261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.893565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.893830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.896737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:25.898462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:25.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:25.908870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:25.909078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:25.929364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:25.930629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:25.932611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:25.932611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:25.932611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:25.932611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 433/867] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:25.936495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.998712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 435/868] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:26.043659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc7d89-bea9-43f9-8def-df31838d9a5e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.093250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.093957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.094175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.094481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.095150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.095451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.096883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f1e8b-9643-4144-9576-b0da34c0ab15 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.111783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 434/869] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.117557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.194114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1b9b5-8f2e-4d50-8680-7e34cd3e4ab4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.194463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1b9b5-8f2e-4d50-8680-7e34cd3e4ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.194677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1b9b5-8f2e-4d50-8680-7e34cd3e4ab4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.196383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 435/870] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:26.201941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.202204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.202523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.210955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.223399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:26.224612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.241576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8867-df83-4730-a1b8-37c66c124411 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.286837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.287709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.288170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.288568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.297760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.299730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.304711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 436/871] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 13 19:45:26.305396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.307485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:26.322142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:26.323365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:26.324470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:26.326761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 436/872] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:26.333398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.441984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1712d7-f4c7-4814-a29d-afc2c04cd9ee None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.452322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ff9f9-6b2f-4dbd-ac1d-446ec77c4040 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.484154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.485134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.485407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.485670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.486261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.487834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.490558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.499773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.500864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.501311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.501848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.502745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:26.503112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:26.504071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:26.504894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 437/873] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:26.505963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.507637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.510478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.532348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 437/874] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.534127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.606054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.652084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91ee36-f885-40c6-a082-0f5566095cda None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.653092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3cde9-b02f-48aa-a591-4c50e74f537e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.691908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.692621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.692863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.693097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.693885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=398ef52b1f264cab85425daaa03e2b6a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.694201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.695360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.695686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.695984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.696261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.696530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.697166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.697635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.699176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.702453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 438/875] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/role_assignments?user.id=398ef52b1f264cab85425daaa03e2b6a&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.703462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.708662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 438/876] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.714642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.787455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.831735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7cdaf-1ac3-4f51-88c1-66151eda598f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.832270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d668658-ecac-445a-b053-4c856bd901d5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.862039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.875701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.876348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.876631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.876901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.877574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.877915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.878574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.878928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.879036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.879280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.879957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.880373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.881679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e04fc4-037b-41aa-acb3-79af8d6c2c68 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.894656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 439/877] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.894656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97552b-bf6f-461d-b9b7-17a2dd7618d4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.894656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97552b-bf6f-461d-b9b7-17a2dd7618d4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.894656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97552b-bf6f-461d-b9b7-17a2dd7618d4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.894656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 440/878] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:26.903435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824f97c-6961-4e9b-a082-ba4b6cfd314c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.903779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824f97c-6961-4e9b-a082-ba4b6cfd314c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.904029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824f97c-6961-4e9b-a082-ba4b6cfd314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.904304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 439/879] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:26 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:26.934404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f824f97c-6961-4e9b-a082-ba4b6cfd314c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:26.936789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f824f97c-6961-4e9b-a082-ba4b6cfd314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.074225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 441/880] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:26 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:27.348131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:27.458311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737290ef-a15f-4e0a-b390-1227cd5bc5ab None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:27.495092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:27.495734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.496128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.496463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.497152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:27.498570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bea75d-9741-44c5-a1fc-a6f339c404a4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:27.517299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 440/881] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:27 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:29.164517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69b25-c7f8-4ef3-a72e-e2a455902a52 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.165243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69b25-c7f8-4ef3-a72e-e2a455902a52 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.165243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69b25-c7f8-4ef3-a72e-e2a455902a52 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.166741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 442/882] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:29.172549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.173226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.173226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.193205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:29.194512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:29.194806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303c5da-e7fc-4389-9862-2f501e05dbc0 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.195072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303c5da-e7fc-4389-9862-2f501e05dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.195315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303c5da-e7fc-4389-9862-2f501e05dbc0 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.196862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 443/883] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:29.202470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.202824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.203057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.221292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:29.222515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:29.237507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:29.274081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 441/884] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:29.286776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:29.297821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 444/885] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:29.548424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:29.558406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:29.669390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bad17-53b9-433b-be26-3b54a15b7b6b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:29.670888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796ecce-9fdb-42bf-b885-3064ec5be81e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:29.710264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:29.710958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.711232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.711496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.712081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:29.712574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:29.713356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.713481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:29.713953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.714386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.715193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:29.716723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574cc972-ed30-43ae-8c1f-e5d5710cf519 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:29.722881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:29.724416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:29.725805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:29.729183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 445/886] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Sep 13 19:45:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:29.730122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:29.746627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 442/887] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:29 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:29.750708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3247-aa4f-475d-be1f-678313e9c8a8 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.751118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3247-aa4f-475d-be1f-678313e9c8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.751463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3247-aa4f-475d-be1f-678313e9c8a8 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.753122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 443/888] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:29.762400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7295e7-8bce-4287-a7cf-c935317c91cc None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.762400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7295e7-8bce-4287-a7cf-c935317c91cc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.762400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7295e7-8bce-4287-a7cf-c935317c91cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.779163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2a7295e7-8bce-4287-a7cf-c935317c91cc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:29.780570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2a7295e7-8bce-4287-a7cf-c935317c91cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:29.811189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:29.812979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:29.879523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06429b55-fc57-42d3-aa8c-513ffe7e0b1f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:29.883136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 444/889] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:29.884084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038418c-0abf-4d42-a93b-6564d3dd8d34 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.884367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038418c-0abf-4d42-a93b-6564d3dd8d34 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.884632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038418c-0abf-4d42-a93b-6564d3dd8d34 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.886287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 445/890] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:29.892081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.892400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.892628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.912523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:29.913762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:29.941170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:29.941786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.942064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.942411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.943069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:29.943374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:29.944763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:29.950411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 446/891] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:29.956099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:29.999985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 446/892] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:30.066744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a1e8-12a7-4648-81db-29b526b378c5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.078101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.104767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.105799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.106296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.106784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.107703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:30.108196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.109765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.116767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 447/893] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:30.122533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.143176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.230503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.247697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739adaa-ed50-4047-9353-18ed51e19c76 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.264828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c464339-a882-4645-8dbf-642f239c3b1b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.285702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.286344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.286645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.286918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.287636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] RBAC: Adding query filter params (interface=public, service_id=1418c6a242e1482e94f7b998479df122, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:30.287946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.289348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.298725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 447/894] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/endpoints?service_id=1418c6a242e1482e94f7b998479df122&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:30.299554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.307577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.308176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.308392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.309149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.323257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:30.328641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:30.329575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:30.330426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 448/895] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:30.331108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.368324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.424680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4308-8ed1-48aa-9611-6b1f7ae77863 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.437038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55811572-a2e8-4da4-97e3-4d123c17cd85 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.466313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.467031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.467334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.467546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.468286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.469545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.474309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.475033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.475323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.475597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.476216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.477593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:30.478994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:30.480091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:30.480904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 448/896] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:30.481655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.494479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 449/897] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:30.494479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.504493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.618883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f957e42-b044-4d41-986d-b54e695ba039 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.633113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.637578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63096e3b-f210-46c1-93ac-76f935a5781e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.663196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.664104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.664626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.665113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.666008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:30.666578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.668601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.678838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 449/898] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:30.684679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.685299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.685947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.686228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.686557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.687210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:30.687519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.688908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.695682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 450/899] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:30.703295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.787781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.818450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308939fa-58d9-4ff8-9789-8a226ec62649 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.826004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393215b-ddaf-4893-8401-8e1501197630 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:30.860885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.861536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.861807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.862091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.864835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bd029-157b-4974-871d-84fc65632852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:30.865556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd029-157b-4974-871d-84fc65632852 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:30.865840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd029-157b-4974-871d-84fc65632852 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:30.866113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bd029-157b-4974-871d-84fc65632852 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:30.866778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bd029-157b-4974-871d-84fc65632852 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:30.867085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bd029-157b-4974-871d-84fc65632852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.868463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bd029-157b-4974-871d-84fc65632852 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.874307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:30.875791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:30.878463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 451/900] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:30.880918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bd029-157b-4974-871d-84fc65632852 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:30.895790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:30.896775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:30.897651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:30.897651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:30.897651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:30.897651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 450/901] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:30.903707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:30.912960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:30.999566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bd029-157b-4974-871d-84fc65632852 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.013843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2839b3-6b0c-46c4-bf7a-869877911a5b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.041147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.042081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.042575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.042988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.044333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] RBAC: Adding query filter params (interface=public, service_id=ed6a9f26fde14bf38c9d74c7f338fa86, region_id=RegionOne) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:31.044936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.046667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.052681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:31.058283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.058759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.058936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.059044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 452/902] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/endpoints?service_id=ed6a9f26fde14bf38c9d74c7f338fa86&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:31.059216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.059886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:31.060198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.060533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.061672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.082698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 451/903] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:31.083688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.177858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:31.191002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3043285-bb17-46ba-9964-900aad8629ed None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.191605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935305f0-8f7f-4c60-98c2-79f9ec64b084 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.228861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.229160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.229910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.230121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.230228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.230537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.231240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.231642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.232395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.233931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.238921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.240757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.241467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 453/904] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:31.255304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:31.256471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:31.257373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:31.259392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 452/905] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:31.265621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.385557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8131acc-877f-483b-a2f9-1157141f6c17 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.434248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.435163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.435441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.435712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.436548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:31.436853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.438402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.459478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 454/906] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:31.466400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.523161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:31.595668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d7bc2-4737-415f-b180-6d1d3a0e0fa6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.631602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.632230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.632511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.632787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.706829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.708668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:31.764072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:31.819939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 453/907] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:31.825611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.841136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:31.942800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2894a6-23d6-4a4d-9e49-d39a33c58ecf None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.979648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.980298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.980578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.980850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.992042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:31.993581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-ceed6c01-cab5-4bfb-922f-5f2b23e115d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:32.009541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:32.010595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:32.011465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:32.011465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:32.011465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:32.011465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 455/908] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:32.017024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.038054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37835-2c22-4bbe-98ac-f64a12381570 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.038392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37835-2c22-4bbe-98ac-f64a12381570 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.038630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37835-2c22-4bbe-98ac-f64a12381570 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.040243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 456/909] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:32.045513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.045793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.046068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.064437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:32.065677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:32.139899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.153148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47c052-8bba-4812-ade4-eaafa2fd5d5f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.154397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 457/910] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:32.189171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.190101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.190549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.190974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.191803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:32.192260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.193772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.206124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 454/911] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:32.211867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.323337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df7e63-1895-4aee-a9e1-1641f60c953a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.359327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.360263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.360743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.361193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.369104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.370785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:32.382821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:32.384004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:32.385032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:32.387077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 458/912] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:32.395519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.407628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.419331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.493312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.503165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9af81-a653-4250-9434-fe39fbc3bb05 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.548831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.549707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.550306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.550754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.552212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:32.552734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.554165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d04404-43b0-4fa4-85ea-236228422d4e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.554599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.575337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 455/913] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:32.581443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.603106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.603759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.604038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.604398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.605090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.606682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:32.616741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:32.617756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:32.618600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 459/914] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:32.626808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.677773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.718946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e45daf-d1f0-495e-b558-f0acdcba4181 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.755270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.756158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.756436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.756618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea669b3-1fbe-4ef9-b7e5-3ae21bd01aaf None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:32.756742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.757428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] RBAC: Adding query filter params (role.id=3e1e2917a95f4a24bbedde5b7a366f6c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=398ef52b1f264cab85425daaa03e2b6a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:32.757730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.759207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e025cdb-cf08-4f9e-9d55-b1aa1d3d8887 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.764986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 456/915] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/role_assignments?user.id=398ef52b1f264cab85425daaa03e2b6a&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=3e1e2917a95f4a24bbedde5b7a366f6c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:32.766190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2564-bc00-4bd3-a7c2-d3b59a819ee5 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.766504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2564-bc00-4bd3-a7c2-d3b59a819ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.766780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2564-bc00-4bd3-a7c2-d3b59a819ee5 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.767990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 457/916] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:32.773873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.774225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.774466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.778603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.803020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:32.803987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:32.804476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:32.804924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:32.805754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:32.806240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.806678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:32.807979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:32.808392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.815229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 460/917] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:32.821150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:32.914685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 458/918] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:32.962114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e758a7-adec-453f-8993-da7e694a6c55 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.000532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.001620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.002176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.002676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.003514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:33.004003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.005640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.011843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 461/919] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:33.019589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.085343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:33.128143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f217784-6e40-454b-9ec5-7d2cda258fc1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.164371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.165068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.165284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.165555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.166264 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] RBAC: Adding query filter params (interface=public, service_id=2c9b98effb1342f38352c5b4afc02fcb, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:33.166659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.168048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.173903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 459/920] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:33 2022] GET /identity/v3/endpoints?service_id=2c9b98effb1342f38352c5b4afc02fcb&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:33.179721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.212300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.233275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:33.298520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a72dc8-7006-44c0-8d3f-72254c741e93 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.335008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.336324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.336778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.337256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.338128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.339799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef8dc-a7e0-421e-bd93-1e9e6f9aa4a2 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.340064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.346981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 462/921] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:33 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:33.377984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:33.390674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.391807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.392297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.392838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.393658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.395456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:33.406139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:33.407606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:45:33.408473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:33.411947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 460/922] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Sep 13 19:45:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:33.419732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.544457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29e169-aca6-4fc3-b9e4-ea6179bef48d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.559354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:33.589953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.590631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.590908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.591183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.591824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:33.592126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.593515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994cd8-6ac8-49b3-a15c-1880d6bbcb65 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.599120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 463/923] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:33.604965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.683386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf587-9dfc-4325-9554-84c03fa7e9c7 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.683563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf587-9dfc-4325-9554-84c03fa7e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.683836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf587-9dfc-4325-9554-84c03fa7e9c7 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.685183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 464/924] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:33.691592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.691878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.692630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.717469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:33.718875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:33.726857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f9d39-b309-49c7-b2d7-b7d95868280e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.774499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.775838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.776417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.776890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.778001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:33.779610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.782387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.795631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 461/925] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:33.797212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:33.808338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 465/926] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:33.860436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:33.909449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28125b9c-65ce-437e-a6cd-dc4356b6bded None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:33.946220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:33.946796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:33.947078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:33.947351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:33.948111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:33.949526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:33.979420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 462/927] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:34.086631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:34.203472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2a142-faa6-4153-8776-745bbb658dd5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:34.206681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:34.251341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:34.251989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.252268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.252541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:34.253129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:34.254541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:34.264013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:34.265022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:34.265866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 466/928] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:34.271164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:34.380224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d1639-8c86-4cfa-9a2b-93162acdb7dc None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:34.392547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:34.419342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:34.420109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.420286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.420560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:34.421198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:34.421498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:34.422942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:34.428446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 463/929] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:34.434241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:34.541051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6148-d9ea-4bae-a55a-724ed4b481b3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:34.582039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:34.582953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.583321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.583601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:34.584427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:34.584839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:34.586295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c52f7c-3f72-4da4-8347-b5a529b66687 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:34.594004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 467/930] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Sep 13 19:45:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:34.600017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:34.711057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9a154-b3ed-49c4-83c2-26c368457455 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:34.746444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:34.747419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.747931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.748389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:34.749263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:34.750860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64651efa-fcd6-48af-b053-338907df0db7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:34.774868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 464/931] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:34.990711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd585dc-ce0e-412f-92ce-a7f3d415ba59 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.991101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd585dc-ce0e-412f-92ce-a7f3d415ba59 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.991278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd585dc-ce0e-412f-92ce-a7f3d415ba59 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:34.992960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 468/932] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:34.998297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:34.998712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:34.998979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.020074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:35.021361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:35.050146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:35.108656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 465/933] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:35.373503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd585dc-ce0e-412f-92ce-a7f3d415ba59 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:35.409250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:35.491045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd585dc-ce0e-412f-92ce-a7f3d415ba59 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:35.530944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:35.531591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:35.531865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:35.532132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.547100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:35.552038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:35.552955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:35.553764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 469/934] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:45:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:35.559327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:35.663827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da5636-0b5e-4570-bacd-20a6bfd66b6b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:35.703968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:35.704610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:35.704888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:35.705156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.705853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:35.706150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:35.707821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7921f69-f54d-4064-ac49-aa58711332bb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:35.722176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 466/935] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:45:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:35.728042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:35.808022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493680-4f6b-45d7-9869-377251272bf1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:35.808312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493680-4f6b-45d7-9869-377251272bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:35.808782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493680-4f6b-45d7-9869-377251272bf1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.810465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 467/936] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:35.816551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:35.817074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:35.817493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.833961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:35.837447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:35.839093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:35.843487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a58e36-78e9-440d-94ac-76d461686b42 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:35.854873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:35.883240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:35.883884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:35.884160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:35.884429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:35.895469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:35.896949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:35.911322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:35.912520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:35.913694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:35.913694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:35.913694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:35.917966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 470/937] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:35.918895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:35.921063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 468/938] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:36.032329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b1755-57fa-4020-9678-a7bf153776c3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.065990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.066640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.066919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.067191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.067836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:36.068140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.069561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.078521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.082403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 471/939] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:36.087488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.189683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09e769-2f0b-4fc8-ab1c-52dc99d14bfd None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.198183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.235262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.236294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.236889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.237464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.246251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.248157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:36.261914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:36.262984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:36.263913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:36.271680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 469/940] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:36.273121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.287453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.323860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a47c7-5e85-436e-82b5-149c1ed4098f None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.362922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.363567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.363842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.364473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.365056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.366720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:36.378388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:36.379389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:36.380226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 472/941] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:36.387480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.389565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3549f5-f5ec-4d6e-a382-98676f75d41c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.427528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.428145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.428424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.428693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.429421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:36.429723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.431192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a746d0c-c4bf-4a30-b602-4bbc91dc18ff None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.451618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 470/942] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:36.451618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b902b3-0560-4711-b627-724b135c7e5e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.451618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b902b3-0560-4711-b627-724b135c7e5e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.451935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b902b3-0560-4711-b627-724b135c7e5e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.454501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 471/943] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:36.456079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b902b3-0560-4711-b627-724b135c7e5e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.466240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.508132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a52dcb-0d61-452d-9df6-da071cfa7879 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.529017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.544201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.544838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.545117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.545358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.545988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:36.546287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.547685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa8ad-0368-40a7-9b83-089c353aa802 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.553054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 473/944] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:36.557981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.557981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.557981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.576629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:36.577833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:36.582186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b902b3-0560-4711-b627-724b135c7e5e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.618393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.619178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.619613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.619943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.630681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.632236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:36.647048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:36.648000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:36.648890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:36.648890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:36.648890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:36.648890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 472/945] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:36.649667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.658405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 474/946] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:36.658405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.729987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.774215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c748-ba51-4d5e-ad20-2c57c71ba415 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.787163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4b67d-5394-4c2c-acc4-ceeb8e0777fe None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.797431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:36.828625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.829072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.829232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.829513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.829781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.830269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.830434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:36.830739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.831000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:36.831459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:36.832128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.832399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:36.832848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:36.834938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:36.838413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 473/947] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:36.844686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.859269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 475/948] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:36.865347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:36.960761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1d4ae-f052-4ddb-ae8a-e50e84ccc149 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.984925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e51c76-5fdf-461c-8962-cab725dd23b5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:36.998879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:36.999506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:36.999774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.000038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.000749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.002134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.004004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:37.030632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 474/949] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:37.032002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.040518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.041346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.041596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.041767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.053156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.055355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:37.068950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:37.069971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.070918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 476/950] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:37.077800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.112431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:37.202245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7c76-3f0c-4aa5-9b9b-0c6cb28a48e0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.214229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6798b9-f74f-4faa-b90e-234f8a4b96d3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.251514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.252086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.252361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.252631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.253213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.254767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.260498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.261123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.261397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.261662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.262354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:37.262705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.264130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec098b-eb93-4a77-9b65-f159286d8bd1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-cfa900c8-44d6-447e-bb47-a741ba32aa1e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:37.264791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:37.265931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:37.266870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 475/951] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:37.268257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6262008-4810-45c2-8224-431a9f86868a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.268536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6262008-4810-45c2-8224-431a9f86868a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.268802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6262008-4810-45c2-8224-431a9f86868a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.271371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 476/952] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:37.276257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6262008-4810-45c2-8224-431a9f86868a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.280255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 477/953] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:37.282329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.282635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.282905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.302603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:37.303803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:37.337110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:37.381748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 478/954] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:37.386696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.391059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6262008-4810-45c2-8224-431a9f86868a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.394775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:37.436901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.437545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.437818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.438084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.438748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:37.439095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.440626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.451088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 477/955] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:37.452112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.503044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236118c5-f318-411a-b1fd-b4f4cf52eb92 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.551042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.552583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.553054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.554407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/398ef52b1f264cab85425daaa03e2b6a/roles/3e1e2917a95f4a24bbedde5b7a366f6c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.558569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=398ef52b1f264cab85425daaa03e2b6a, role_id=3e1e2917a95f4a24bbedde5b7a366f6c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.561332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.588268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a3c5-9aeb-4e81-9782-17aaedb65ec1 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.589602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 479/956] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:37 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/398ef52b1f264cab85425daaa03e2b6a/roles/3e1e2917a95f4a24bbedde5b7a366f6c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:37.630959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.631897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.632315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.632727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.633523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:37.633979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.635752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.641322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 478/957] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:37.669056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.702021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:37.800987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92b25e-2914-4fdd-b223-42d943bff26c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:37.841111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:37.841737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:37.842013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:37.842281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:37.842987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:37.844449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:37.847213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING keystone.server.flask.common [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 19:45:37.868253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.sql.core [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:45:37.868253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:45:37.868253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [parameters: {'id': '0827c4531c8a47bf8e6f5e5250f7914c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:45:37.868253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81576) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0827c4531c8a47bf8e6f5e5250f7914c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0827c4531c8a47bf8e6f5e5250f7914c. Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:45:37.884873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application result.read() Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 19:45:37.885934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application yield resource Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise self.value Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application session.commit() Sep 13 19:45:37.887155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:45:37.888039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:45:37.888932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application result.read() Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:45:37.889735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [parameters: {'id': '0827c4531c8a47bf8e6f5e5250f7914c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.891639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:37.893002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0827c4531c8a47bf8e6f5e5250f7914c. Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:37.894576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 480/958] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Sep 13 19:45:37 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:37.897361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:37.952218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:38.036983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a5492-96b7-4d9b-bc26-7047ac499f53 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:38.073747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:38.074347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:38.074664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:38.074993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:38.082515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:38.083992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-fbefc8d3-3387-4249-ab7d-c4af28167a0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:38.096302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:38.097228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:38.098583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 479/959] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:38.104064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:38.189756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:38.209935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122f3ac-fc6a-4daa-8afa-f970e7940667 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:38.246734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:38.247366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:38.247647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:38.247921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:38.248600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:38.248904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:38.250592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7af8f-f7a9-4368-8e58-5509405d668b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:38.278193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 481/960] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:39.382332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02fb9a-b285-4dd5-9229-ee2e052b9b33 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.382906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02fb9a-b285-4dd5-9229-ee2e052b9b33 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.383345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02fb9a-b285-4dd5-9229-ee2e052b9b33 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.386212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 480/961] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:39.395928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9e969-bce4-40aa-a1ef-e392e5c97b59 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.396500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9e969-bce4-40aa-a1ef-e392e5c97b59 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.396924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9e969-bce4-40aa-a1ef-e392e5c97b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.427739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-58b9e969-bce4-40aa-a1ef-e392e5c97b59 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:39.429593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-58b9e969-bce4-40aa-a1ef-e392e5c97b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:39.505821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 482/962] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:39.770158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02fb9a-b285-4dd5-9229-ee2e052b9b33 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:39.858760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a938-c9d7-4e58-ae29-2adb9a8c8765 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.859122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a938-c9d7-4e58-ae29-2adb9a8c8765 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.859305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2a938-c9d7-4e58-ae29-2adb9a8c8765 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.861000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 483/963] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:39.867054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f728-54f0-4058-9881-c680ab225943 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.867328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f728-54f0-4058-9881-c680ab225943 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.867604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3f728-54f0-4058-9881-c680ab225943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.874183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:39.887492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02fb9a-b285-4dd5-9229-ee2e052b9b33 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:39.889531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-10b3f728-54f0-4058-9881-c680ab225943 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:39.891047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-10b3f728-54f0-4058-9881-c680ab225943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:39.934413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:39.935075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.935357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.935628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.936227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:39.937617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:39.950243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:39.951494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:39.952330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 481/964] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:39.957381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:39.972098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:39.982697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 484/965] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:39.983998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919acd6-2490-4faf-b07f-3735afa30bbb None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.984278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919acd6-2490-4faf-b07f-3735afa30bbb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.984549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919acd6-2490-4faf-b07f-3735afa30bbb None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:39.985935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 485/966] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:39.991688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:39.991965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:39.992236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.011869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:40.013047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:40.071669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfaacb5-9290-4eb7-ae43-a299a7a23e4f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.098445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 486/967] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:40.119933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.120921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.121427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.121892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.122763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:40.123265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.124972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.130875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 482/968] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:40.136443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.241033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444455e-3581-4192-97ef-97f5f355b0cf None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.259269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.276682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.281322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.284050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.284844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.285124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.285396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.286043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:40.286347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.288327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.298438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 487/969] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:40.300940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.402790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c000-7637-4c07-8dae-dad5c3215868 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.412366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da341c8d-e4bd-4a7c-9c01-fa1fcc886fe2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.442651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.443313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.443593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.443862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.450262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.450992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.451277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.451551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.452485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] RBAC: Adding query filter params (interface=admin, service_id=2c9b98effb1342f38352c5b4afc02fcb, region_id=RegionOne) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:40.452799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.454240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.458234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:40.465168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 488/970] 10.222.0.52 () {56 vars in 1249 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/endpoints?service_id=2c9b98effb1342f38352c5b4afc02fcb&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:40.465168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:40.465421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:40.466249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:40.467103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:40.469858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 483/971] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:40.470745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.530651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.561821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.573184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ec406-6663-49ba-a348-120fae42c7fa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.585300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5269-5f3c-483f-8d7b-b53b092cba3d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.607958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.608580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.608856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.609123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.609769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.611166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.620948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.621574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.621851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.622127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.622767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.624141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a623a5b-632c-4543-a798-06e75afc15fd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.627278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 489/972] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:40 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:40.628089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.642404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 484/973] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:40.642404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979bfff-f1b0-4f01-b80b-b55b018415bc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.642404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979bfff-f1b0-4f01-b80b-b55b018415bc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.642404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979bfff-f1b0-4f01-b80b-b55b018415bc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.642404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 485/974] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:40.649638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.649975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.653220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.670831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:40.672117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:40.783127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840655e-1afa-469a-b542-85a13b5a6714 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:40.786562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 486/975] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:40.815944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.834078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:40.834752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:40.835029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:40.835297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:40.835980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:40.836279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:40.837682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:40.848548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 490/976] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:40.854402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:40.926969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:40.964281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f382bf-23af-42b8-b83a-958f2242ed8d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.002575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.003341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.003628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.003901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.015157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.016646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:41.032361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:41.033393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:41.034272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:41.034272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:41.034272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.034272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 487/977] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:41.039465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.071123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.161750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4ae11-a099-4b73-82c6-cf4caa004b33 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.190099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62db9-e1bd-4cc3-9ab1-e9577c295164 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.198892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.199619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.199898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.200169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.200825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:41.201119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.202650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.215948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:41.217071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 491/978] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:41.222495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.237673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.238779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.239511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.239929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.240710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.243198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.247112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING keystone.server.flask.common [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 19:45:41.266140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:41.269216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.sql.core [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:45:41.269216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:45:41.269216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [parameters: {'user_id': 'a359ec6edc354ef0adc8d1e6f34dd39b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:45:41.269216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result.read() Sep 13 19:45:41.280997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:45:41.281953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application yield resource Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise self.value Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application session.commit() Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:45:41.282972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:45:41.283819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:45:41.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application result.read() Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:45:41.286480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a359ec6edc354ef0adc8d1e6f34dd39b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:41.287880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 19:45:41.289559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 488/979] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Sep 13 19:45:41 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:41.291680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.361043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad7d2d-cf23-4015-849a-6d7e22ae7cfa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.403254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.404026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.404421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.404691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.410575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e331b7-69e7-45b7-8cc7-c9c733d71325 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.414400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.415952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:41.430298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:41.431357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:45:41.432313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:41.436701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 492/980] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:41.438289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.458272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.459761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.460547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.461233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.470457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.472392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:41.485489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:41.486523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:45:41.487552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:41.490571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 489/981] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:41.496671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.556368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:41.565945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:41.576637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0c88-7e70-4fc9-87f7-6ec59c102914 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.619262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.620178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.620604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.621017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.621928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:41.622599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.624562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.641083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c1cde-504b-4cf8-9ed6-39f289969578 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.650951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 493/982] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:41.653727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:41.686323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.687166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.687716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.688317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.689630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:41.690295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.692399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:41.749762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 490/983] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:41.779102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8663653-c6bb-4ebf-953e-bfde52a1ded1 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:41.818095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:41.818966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:41.819372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:41.819769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:41.847949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:41.901940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:41.903859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.018946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 494/984] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:41 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:42.025172 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:42.135805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:42.140243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99425b2e-5282-4f46-9819-fd204135199d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:42.176246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:42.177054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.177486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.177846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.188618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:42.190207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:42.205648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:42.206943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:42.208121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:42.208121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:42.208121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:42.209296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 491/985] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:42.214923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:42.277188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:42.323080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5da1a5-64db-4296-b17b-415fadc9877a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:42.359796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:42.360563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.360929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.361288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.362055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:42.362472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:42.364297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.376882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 495/986] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:42.382601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:42.492367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27413a8e-d5d0-449d-a7d1-b3eba59ac04f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:42.529862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:42.530422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.530721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.530963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.541213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:42.541213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:42.553790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:42.554966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:42.555986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:42.557003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 492/987] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:42.561282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:42.673241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5f303-2ca2-4cfa-b53c-9cee44eef09b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:42.692381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:42.714059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:42.714716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.714994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.715265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.715975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:42.716280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:42.717728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3920b87-ea42-4041-af3e-fa02b0dab73b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.735189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 496/988] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:42.737175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:42.754052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:42.863664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3234cf-7084-46ce-b047-a20673ea1529 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:42.892530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cd2-1f95-4334-83c5-6fe44ffa5f52 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.892811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cd2-1f95-4334-83c5-6fe44ffa5f52 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.893083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6cd2-1f95-4334-83c5-6fe44ffa5f52 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.894731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 497/989] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:42.900390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.900938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.901383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.908962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:42.909530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.909811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.910112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.910878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] RBAC: Adding query filter params (role.id=3e1e2917a95f4a24bbedde5b7a366f6c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=398ef52b1f264cab85425daaa03e2b6a) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:42.911198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:42.912852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530fb80-ecfc-4277-b47b-1723ee2a1508 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:42.919709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 493/990] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:42 2022] GET /identity/v3/role_assignments?user.id=398ef52b1f264cab85425daaa03e2b6a&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=3e1e2917a95f4a24bbedde5b7a366f6c => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:42.921366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:42.922603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:42.978026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce647c-c3fc-4195-9f41-4c9cbf19f12a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.978285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce647c-c3fc-4195-9f41-4c9cbf19f12a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.978696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce647c-c3fc-4195-9f41-4c9cbf19f12a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:42.980360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 494/991] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:42.986225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:42.986598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:42.986840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.004858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 498/992] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:43.007108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:43.008718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:43.086259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 495/993] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:43.271822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.350114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.358425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.387049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e162a-f270-4866-96f1-86da77d8ba63 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.390241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.425018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.426092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.426398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.426682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.427383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.428754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:43.439347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:43.440415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:45:43.441259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:43.443480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 499/994] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:43.446787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.480460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6dfbd-c501-4ea5-b78c-e37bdde8c8a2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.521189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.522675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.522887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.523157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.523756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.525177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:43.537460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:43.538725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:43.539654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 496/995] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:43.544702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.569158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d1210-cb39-4633-8ef4-d4d5a9f0acaa None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.607667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.608561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.609058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.609332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.609969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:43.610267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.611691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.622327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 500/996] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:43.623412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.626499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.660084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477970da-661d-4b5b-9697-862b097ff957 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.661812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.700423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.701186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.701452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.701728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.702346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:43.702672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.704061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.713977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 497/997] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:43.715201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.742308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687850b-9d00-4db7-a4a0-67591b6d26c6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.780665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.781176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.781461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.781729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.782402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:43.782697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.784072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.789251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 501/998] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:43.795348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.832524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e4b4-432c-4440-bf47-40cb0aecd074 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.867694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.868305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.868583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.868852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.869689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:43.870061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.871579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.873302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.878457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 498/999] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:43.884813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:43.916559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:43.923583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c68b4b-1a0e-4acc-b1be-b9dc7face552 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:43.960909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:43.961526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:43.961800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:43.962066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:43.962787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:43.964567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:43.989973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 502/1000] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:44.001202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5e70c-392f-419e-ad76-5f4727a3e37f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:44.052451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:44.052954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:44.053237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:44.053508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:44.054237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] RBAC: Adding query filter params (interface=public, service_id=68de959b422d4249b04c067995379fca, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:44.054585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:44.056053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713cc3-50f0-409e-8304-ce8d19474b02 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:44.062744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 499/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:43 2022] GET /identity/v3/endpoints?service_id=68de959b422d4249b04c067995379fca&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:44.068752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:44.186247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e2a3f-8d1b-4903-8ca8-e9c5fe9d7fdf None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:44.223780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:44.224414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:44.224696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:44.224976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:44.225596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:44.227011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf74a2-b91b-4ae3-9ff3-e58eebad6c88 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:44.238484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 503/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:44 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:45.238967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e1e-bf2b-4f9b-b7f2-c35e4b7daad3 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:45.239647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e1e-bf2b-4f9b-b7f2-c35e4b7daad3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:45.239647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e1e-bf2b-4f9b-b7f2-c35e4b7daad3 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:45.240993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 500/1003] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:45.247998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:45.247998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:45.247998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:45.267664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:45.270323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:45.293258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:45.361697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 504/1004] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:45.632411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8e1e-bf2b-4f9b-b7f2-c35e4b7daad3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:45.711670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:45.763103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8e1e-bf2b-4f9b-b7f2-c35e4b7daad3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:45.800846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:45.801496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:45.801773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:45.802034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] PATH_INFO: `/v3/services/image` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:45.802649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:45.804061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:45.813151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:45.814713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:45.815616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 501/1005] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Sep 13 19:45:45 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:45.822831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:45.931958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c6e21-c0d3-4776-a6c3-c3ba8acb1860 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:45.970420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:45.971535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:45.971535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:45.971804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:45.972377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:45.972617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:45.974723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7a1ec-a622-419d-9aa2-e44450f9c7d2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:45.987561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 505/1006] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:45.997821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:46.149025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e677269-4551-454a-8272-93e4ff00396e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:46.183206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:46.200541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:46.201400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.201787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.202161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.203159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] RBAC: Adding query filter params (type=image) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:46.203593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:46.205743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531117b-b503-49dc-8d5e-fe03f229dc0e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:46.218898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 502/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:46.235906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63bfa-9520-43a9-abc7-10adee8865f9 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.236233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63bfa-9520-43a9-abc7-10adee8865f9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.236442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb63bfa-9520-43a9-abc7-10adee8865f9 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.242815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 506/1008] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:46.249120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84989-158e-4dac-b345-d7a012a186e3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.249120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84989-158e-4dac-b345-d7a012a186e3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.249120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84989-158e-4dac-b345-d7a012a186e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.272450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b2c84989-158e-4dac-b345-d7a012a186e3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:46.274236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b2c84989-158e-4dac-b345-d7a012a186e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:46.390440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 503/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:46.650883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb63bfa-9520-43a9-abc7-10adee8865f9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:46.664952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459849a-fd8d-4399-9870-7a53a0270db4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.665300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459849a-fd8d-4399-9870-7a53a0270db4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.665519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459849a-fd8d-4399-9870-7a53a0270db4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.667283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 504/1010] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:46.670031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:46.673175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.673470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.673746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.695257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:46.696491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:46.782828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 505/1011] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:46.823217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb63bfa-9520-43a9-abc7-10adee8865f9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:46.864572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:46.865398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:46.865868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:46.866324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:46.868089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:46.870583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:46.872430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 507/1012] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:46.878094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:46.927297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:46.989510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378df8e-1344-4f2b-978b-f1b8711a616c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:47.030977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:47.031626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:47.031906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:47.032179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:47.032856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:47.034413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:47.082887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 506/1013] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:46 2022] POST /identity/v3/projects => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:47.085265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:47.194848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:47.217854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9298e9a-0716-40d0-a52d-ef82b3f19f85 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:47.256461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:47.257124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:47.257408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:47.257679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:47.258278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:47.259882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:47.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:47.271147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:45:47.271995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:47.274812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 508/1014] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:47.277349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:47.391577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:47.397895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a699d-91eb-4048-942b-afe3db3c9d03 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:47.443138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:47.443960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:47.444349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:47.444716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:47.445765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] RBAC: Adding query filter params (name=compute) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:47.446246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:47.448623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:47.457064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 507/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:47.462790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:47.594076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5c4f-8cbf-4753-9412-49a09ba9948a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:47.633842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:47.634542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:47.634806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:47.635076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:47.635722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:47.636023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:47.637476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8a447-f06c-4267-9533-1d92196b2101 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:47.643364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 509/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Sep 13 19:45:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:47.649570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:47.767649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541bd6c-e6df-4665-aca2-c2615fc470ab None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:47.805319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:47.805962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:47.806229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:47.806608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:47.807315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:47.808715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a2588-02b6-4210-8d2c-739d6d907154 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:47.831104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 508/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:48.482779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0f06a-9eec-4bf4-bf9a-8d22c6edf337 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:48.483316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0f06a-9eec-4bf4-bf9a-8d22c6edf337 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:48.483316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0f06a-9eec-4bf4-bf9a-8d22c6edf337 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:48.484893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 510/1018] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:48.491284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d0f2-dd1a-494b-8bf8-71e660f01393 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:48.491776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d0f2-dd1a-494b-8bf8-71e660f01393 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:48.491776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2d0f2-dd1a-494b-8bf8-71e660f01393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:48.511761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a2c2d0f2-dd1a-494b-8bf8-71e660f01393 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:48.512955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a2c2d0f2-dd1a-494b-8bf8-71e660f01393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:48.528934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:48.614401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 509/1019] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:48 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:48.876095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0f06a-9eec-4bf4-bf9a-8d22c6edf337 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:48.890577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:49.009254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0f06a-9eec-4bf4-bf9a-8d22c6edf337 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:49.045452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:49.046075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:49.046306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:49.046635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:49.047292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:49.048685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5281d-b128-48c8-8188-ac62d12ed24a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:49.067092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 511/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:45:48 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:45:49.461457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83cd4-9d3c-4313-bbd4-39ec9f45a9fc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:49.461725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83cd4-9d3c-4313-bbd4-39ec9f45a9fc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:49.462005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca83cd4-9d3c-4313-bbd4-39ec9f45a9fc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:49.463468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 510/1021] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:49.469394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:49.469683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:49.469962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:49.489489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:49.490717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:49.562133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:49.582408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 512/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:49.847421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83cd4-9d3c-4313-bbd4-39ec9f45a9fc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:49.961303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca83cd4-9d3c-4313-bbd4-39ec9f45a9fc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:50.005215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:50.005871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:50.006151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:50.006432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:50.007584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:50.009158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd82f2-8aac-44a4-b9e6-d4c651c354ae None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:50.011013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 511/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:50.016871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:50.137145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4835051-640e-46cb-acfd-5acf8a13f133 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:50.182551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:50.183160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:50.183450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:50.183714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:50.184378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:50.185846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7c6bb-f306-4575-a9a1-5dcd52d4b153 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:50.217684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 513/1024] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:45:50 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:45:50.280132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658821-b418-4050-816f-ee52f743fa0e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:50.280427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658821-b418-4050-816f-ee52f743fa0e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:50.280755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb658821-b418-4050-816f-ee52f743fa0e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:50.282038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 512/1025] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:50.287677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:50.287972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:50.288245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:50.308768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:50.309993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:50.368942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:50.394512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 514/1026] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:50.667175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb658821-b418-4050-816f-ee52f743fa0e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:50.683175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:50.782025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb658821-b418-4050-816f-ee52f743fa0e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:50.819378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:50.820017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:50.820295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:50.820579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:50.832761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:50.837776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:50.839141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:50.843285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 513/1027] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:45:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:50.846946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:50.958992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e27ff-cf12-4676-9aee-fdd789d85e59 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.000816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.001746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.002225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.002713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.003581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:51.004056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.005609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.016128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 515/1028] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:45:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:51.022275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.103139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:51.141188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c99b1-67e3-4f36-93de-6a5d469820c8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.180231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.180904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.181233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.181505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.194557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.196140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-931ab9c7-08a8-45df-b785-91a174ae5938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:51.212608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:51.214498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:51.216124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:51.216124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:51.216124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:51.218490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 514/1029] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:51.220012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.271245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:51.289996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f453c-036b-4264-896c-bc02cdd1a1e0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.290314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f453c-036b-4264-896c-bc02cdd1a1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.290668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f453c-036b-4264-896c-bc02cdd1a1e0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.292038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 515/1030] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:51.297236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.297504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.297782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.316395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:51.317594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:51.333903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483537f4-f677-4046-812a-7ea29abb32dd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.376845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.377750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.378165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.378611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.379534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:51.380046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.381891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.392790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 516/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:51.396885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 516/1032] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:51.402650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.468237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:51.519845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986110e-bc78-4751-8efb-fe9e6221c036 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.562299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.563153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.563430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.563697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.575069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.576556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:51.589811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:51.591326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:51.592352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 517/1033] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:51.597767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.669856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.711265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355830cd-4f25-4acc-b4fa-140035c4c0e9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.715247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:51.754300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.754899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.755182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.755457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.756197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:51.756501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.758953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.779393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 517/1034] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:51.785060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.800174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898a429-b5bb-4772-9ce8-c4989620ad05 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.816016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:51.844057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.844923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.845049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.845220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] PATH_INFO: `/v3/services/image` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.845809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:51.847219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:51.861533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:51.862632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:51.863879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 518/1035] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:51.870628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:51.901343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847d382-a919-4741-92ea-9e305ac18d81 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:51.937871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:51.938573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:51.938851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:51.939122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:51.971768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.014007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.015422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.027016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870a3bf-e293-4862-9aaa-e381d83b9a40 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.089146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.089767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.090201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.090510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.091654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:52.091962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.093352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.098733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 519/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:52.104862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.134687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 518/1037] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:45:52.135924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.151993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.202227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.215475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5c59a-123f-4f01-8834-da51fe4be2d8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.243895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5cfc0-9b8c-4d42-be03-e60ac07f799c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.253457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.254088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.254389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.254666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.255298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] RBAC: Adding query filter params (type=image) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:52.255602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.256980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.262420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 520/1038] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:52.268112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.291404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.292051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.292326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.292597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.303626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.305175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:52.320534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:52.321534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:52.323023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:52.323023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:52.323023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:52.323023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 519/1039] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:52.327356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.382812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136e93c-4260-44b4-b5e9-56f7fb525a8e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.390971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.421117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.421743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.422073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.422335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.423024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] RBAC: Adding query filter params (interface=public, service_id=03faa83c733c45edaa42d96954ada23f, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:52.423383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.424777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.435320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 521/1040] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/endpoints?service_id=03faa83c733c45edaa42d96954ada23f&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:52.436051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.436368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b348433-ca31-48a7-8a99-033677d6bc18 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.452422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.477227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.477988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.478395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.478739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.479397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:52.479702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.481141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.494156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 520/1041] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:52.499572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.557994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd8b8b-b608-49ec-95cc-4dfd6d43a4a8 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.595875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.596500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.596782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.597252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.597667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.599084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca46de-fbf1-4709-8190-3da7f5f498d0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.606277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 522/1042] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:45:52.607774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd91d-3462-4bbb-b598-560a5a36750a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.608060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd91d-3462-4bbb-b598-560a5a36750a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.608334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd91d-3462-4bbb-b598-560a5a36750a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.609574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 523/1043] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:52.612603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512193e3-8bf0-40ad-9bb1-81ad59e83591 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.616084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.616372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.616642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.636896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:52.638131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:52.656110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.656784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.657082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.657423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.666186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.667777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.668725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:52.682829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:52.683827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:52.684747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 521/1044] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:52.690353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.716349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 524/1045] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:52.737497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:52.817136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368cb7-3181-4fea-b6e7-05e5f759dc0b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:52.860589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:52.861762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:52.862254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:52.862780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:52.864068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:52.864573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:52.866333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:52.881758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 522/1046] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:52.891729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:52.995366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:53.062794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc28ca-ee46-4bf3-b8c0-5e9c2ba3b9d1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:53.114507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:53.116121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:53.116121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:53.116121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:53.116532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] RBAC: Adding query filter params (role.id=3e1e2917a95f4a24bbedde5b7a366f6c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:53.116630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:53.118039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:53.122909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:53.134420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 525/1047] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=3e1e2917a95f4a24bbedde5b7a366f6c => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:53.134633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9abba4-06f1-42e1-8276-cc2ce817570c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:53.173455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:53.179931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:53.179931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:53.179931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:53.193083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:53.198443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:53.200122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:53.200964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:53.202335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 523/1048] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:53.205979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:53.241756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:53.325066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95058b-afe8-47ce-9b3d-bee13076bf5c None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:53.362013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:53.362659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:53.362935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:53.363567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:53.364566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:53.364811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:53.366193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:53.376359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 526/1049] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:53.382430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:53.494566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd955a1-4cef-4682-a44e-3863b1dcb641 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:53.531462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:53.532101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:53.532379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:53.532679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] PATH_INFO: `/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:53.552861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] RBAC: Authorizing `identity:get_user(user_id=aa4bdffc1c0f4fc1bde7d2a75205d084)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:53.554337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:53.561197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 524/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:45:53 2022] GET /identity/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:53.562320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:53.671927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:53.684834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b95a9e-617d-4eb0-be48-ae3ecff8167a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:53.723108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:53.723862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:53.723990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:53.724277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:53.796602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:53.798096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:53.890621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:53.907432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 527/1051] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:53.908828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:53.925256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:54.021568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83808eef-4c8f-4489-bf50-24ad2313a235 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:54.055977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:54.056657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:54.056927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:54.057192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:54.058314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:54.059780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1866ab8-0888-41b7-95e9-e7b053f27fa3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:54.061604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 525/1052] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:45:53 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:54.067610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:54.170574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13667b59-0a5c-41cb-b6a1-2dc848fbfef6 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:54.205090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:54.205610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:54.205886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:54.206150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:54.206903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=6dc869d1fe474af88431f3bffae94741, user.id=aa4bdffc1c0f4fc1bde7d2a75205d084) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:54.207205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:54.208630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9eb79-2bff-4072-86cd-abe83e9b24c1 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:54.214597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 528/1053] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:45:54 2022] GET /identity/v3/role_assignments?user.id=aa4bdffc1c0f4fc1bde7d2a75205d084&scope.project.id=6dc869d1fe474af88431f3bffae94741&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:54.840567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c223d-a215-4885-a6ef-c55012723aca None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:54.840769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c223d-a215-4885-a6ef-c55012723aca None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:54.841047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c223d-a215-4885-a6ef-c55012723aca None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:54.842486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 526/1054] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:54.848208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857cfc2-a424-420a-ab9c-f2cccb92c7a3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:54.848467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857cfc2-a424-420a-ab9c-f2cccb92c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:54.848761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857cfc2-a424-420a-ab9c-f2cccb92c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:54.867958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d857cfc2-a424-420a-ab9c-f2cccb92c7a3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:54.869194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d857cfc2-a424-420a-ab9c-f2cccb92c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:54.922753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:54.944045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 529/1055] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:55.208836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c223d-a215-4885-a6ef-c55012723aca None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.307313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:55.331146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c223d-a215-4885-a6ef-c55012723aca None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:55.368050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:55.368763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.369044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.369317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.369907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:55.371332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:55.380807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:55.381895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:45:55.382750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:55.386236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 527/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Sep 13 19:45:55 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:55.388044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ae9b-36ef-480d-a849-38c886ef7d5b None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.388521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.388605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ae9b-36ef-480d-a849-38c886ef7d5b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.388693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439ae9b-36ef-480d-a849-38c886ef7d5b None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.390407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 530/1057] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:55.395267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.395544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.395820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.414862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:55.416102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:55.486356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 531/1058] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:45:55.503312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b4554-86e5-4b1d-834f-ecb6f19da1f9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:55.545166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:55.545913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.546274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.546607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.547396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] RBAC: Adding query filter params (name=image) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:55.547688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:55.549174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:55.560534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 528/1059] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:55.561173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.674121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd177123-6e67-4da3-98e2-f0f2b345a18e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:55.688882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:55.712949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:55.713735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.714141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.714580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.715388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] RBAC: Adding query filter params (type=image) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:55.715811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:55.717338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:55.728674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 532/1060] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:55.729764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.743560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:55.750952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.860950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1f645-922f-46c5-9af2-0067721c3c4e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:55.862624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6329bb6-f6f9-4d2e-95b4-6eedbe56a420 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:55.904027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:55.904668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.904946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.905216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.911270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:55.912065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:55.912503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:55.912905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:55.913751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:55.915349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:55.917138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:55.922004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:55.923167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:45:55.924198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:55.928231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 533/1061] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:45:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:55.929089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:55.958589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 529/1062] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:55.981088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.053099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355e9d7-c744-4dcd-8118-a39e072b855f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.098397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.099059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.099342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.099617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.100309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:56.100616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:56.102062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:56.111789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 534/1063] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:45:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:56.118334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.189889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.234406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b0ae-c79b-461c-be90-bda0c98c23dd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.274497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.275213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.275490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.275762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.288171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:56.289659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-44bcc149-d343-4643-86c0-350d3e9eda6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:56.305186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:56.306153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:56.307447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:56.307447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:56.307447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:56.307447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 530/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:56.312466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.370994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef7b-6529-423b-bc35-6bb11dd128b4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.371265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef7b-6529-423b-bc35-6bb11dd128b4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.371555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034ef7b-6529-423b-bc35-6bb11dd128b4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.373143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 531/1065] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:56.378241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.378645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.378926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.398083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:56.399898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:56.420270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.427487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370df8-555d-4e73-aa29-a12c74261a12 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.477361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.477361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.477361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.477361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.478009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:56.478316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:56.479928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:56.483596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 532/1066] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:56.493264 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 535/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:56.498931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.551196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.610325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc568f-dec8-4e8f-817d-7493d3b0a42a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.647130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.647763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.648048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.648314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.655878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:56.657330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:45:56.669480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:56.670607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:56.676890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 533/1068] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:56.678846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.751061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.791391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b72cf3-08f5-41ed-a3de-927903c0e8f5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.828238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.828783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.829067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.829341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.830077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:56.830406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:56.831867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:56.850210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 536/1069] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:56.855977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.866288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0458a0-56d6-407f-8c73-c13628193cbe None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:56.874729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.874952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:56.907896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:56.908645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:56.908922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:56.909254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:56.921964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:45:56.927515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:56.928596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:45:56.929402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:45:56.934196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 534/1070] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:56.935217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:56.985727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a011fdc-1c72-4c20-9256-1a3c30ad0997 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.024395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.025177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.025458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.025830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.037278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.039007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.044536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7a5f3-018f-46d4-b8b3-f757b37a6f73 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:57.055184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:57.056320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:57.057197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:57.057197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:57.057197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:57.057197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 537/1071] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:57.062592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.084026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.084565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.084903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.085180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.085912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:57.086213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.087648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.101723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 535/1072] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:57.103063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.132023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:57.134088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:57.172972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97bda3-ad46-449c-859f-20f15ef32840 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.210640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362d47f-e46c-477e-b651-9d0298e4e08e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.211672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.212528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.212815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.213946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.214646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:57.214957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.216361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.230711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 538/1073] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:57.236185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.253222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.253983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.254142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.254446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] PATH_INFO: `/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.255611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=aa4bdffc1c0f4fc1bde7d2a75205d084)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.257293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.259195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 536/1074] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:45:57 2022] GET /identity/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:57.264849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.341078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54ca39-9115-445b-9dbd-d7a596da341d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.368600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:57.369492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:57.375459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a74ba7-466c-4eb7-a99e-3fc75407b1d2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.379632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.380272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.380546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.380813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.388354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.389811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:45:57.401458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:57.402428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:45:57.403306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:57.407619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 539/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:45:57.408380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.415671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.416526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.416971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.417401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.418569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.420014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.421690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 537/1076] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:45:57 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:57.427573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.521000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8438c9-64a1-44a6-85e8-6bcf80cc2be7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.535325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80513125-dc93-4d06-b967-1750c2c7b7be None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.563276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.563908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.564191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.564452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.565153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:57.565459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.570361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.571988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.573065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.573065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.573646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741/users/aa4bdffc1c0f4fc1bde7d2a75205d084/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.575249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc869d1fe474af88431f3bffae94741, user_id=aa4bdffc1c0f4fc1bde7d2a75205d084, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.576702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a72db-9e94-4bfa-8e95-9129ea9ad91c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.584571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 540/1077] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:45:57.591276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:57.597339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 538/1078] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:57 2022] PUT /identity/v3/projects/6dc869d1fe474af88431f3bffae94741/users/aa4bdffc1c0f4fc1bde7d2a75205d084/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:45:57.653624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:57.720177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd1565-48a5-4758-b308-ae10996a3d5e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:57.759536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374495c-b911-4696-8712-a58548789562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:57.760227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374495c-b911-4696-8712-a58548789562 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:57.760503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374495c-b911-4696-8712-a58548789562 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:57.760757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374495c-b911-4696-8712-a58548789562 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/3e1e2917a95f4a24bbedde5b7a366f6c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:57.763076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374495c-b911-4696-8712-a58548789562 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=463fcb701698473bad12e8101b9018ce, role_id=3e1e2917a95f4a24bbedde5b7a366f6c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:57.764551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374495c-b911-4696-8712-a58548789562 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:57.786419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 541/1079] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:45:57 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/463fcb701698473bad12e8101b9018ce/roles/3e1e2917a95f4a24bbedde5b7a366f6c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:58.181229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8a98-2a1b-4cf5-a680-90ee8e7c9cb1 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:58.181588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8a98-2a1b-4cf5-a680-90ee8e7c9cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:58.181753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd8a98-2a1b-4cf5-a680-90ee8e7c9cb1 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:58.183350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 539/1080] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:58.188579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:58.188833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:58.189138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:58.208177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:58.209361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:58.281707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 542/1081] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:58.544950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8a98-2a1b-4cf5-a680-90ee8e7c9cb1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:58.575025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:58.661003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd8a98-2a1b-4cf5-a680-90ee8e7c9cb1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:58.700311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:58.701036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:58.701215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:58.701514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:58.771826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:58.773286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:58.797501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:58.884882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 540/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:45:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:45:58.890387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:58.928758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:59.000701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fcdb5-f893-43eb-94cf-a420fce516ae None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:59.034925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:59.035565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.035851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.036110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.047049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:59.048745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6ec18a90-7105-4156-a5f1-c24d2d42c912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:45:59.064119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:45:59.065090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:45:59.065987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:45:59.065987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:45:59.065987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:45:59.070215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 543/1083] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:45:59.071513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:59.175099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52043d8-2713-4276-9ae1-356780890062 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:59.199953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:59.212712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:59.213331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.213609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.213870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.214545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:59.214844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:59.216213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0815ee2-2d09-4e27-8ddd-a59041131657 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:59.228445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 541/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:59.767857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af52a5-d3cc-4772-a834-5468e6a3b9e9 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.768117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af52a5-d3cc-4772-a834-5468e6a3b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.768422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af52a5-d3cc-4772-a834-5468e6a3b9e9 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.769788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 544/1085] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:59.775697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b67f7-e7e1-40fa-97da-f68a29c8ba7c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.776165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b67f7-e7e1-40fa-97da-f68a29c8ba7c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.776317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b67f7-e7e1-40fa-97da-f68a29c8ba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.795454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-851b67f7-e7e1-40fa-97da-f68a29c8ba7c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:59.796775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-851b67f7-e7e1-40fa-97da-f68a29c8ba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:59.876281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 542/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:45:59.938680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97b88-348e-4f9e-a528-61304a08b290 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.939232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97b88-348e-4f9e-a528-61304a08b290 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.939232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97b88-348e-4f9e-a528-61304a08b290 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.940754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 545/1087] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:59.946671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:59.946941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:59.947209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:59.967080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:59.968311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:00.040543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 543/1088] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:00.140738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97b88-348e-4f9e-a528-61304a08b290 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.234562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.248872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97b88-348e-4f9e-a528-61304a08b290 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.283758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.284399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.284677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.284947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.295550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:00.300418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:00.301464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:00.302316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 546/1089] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:00.307986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.321668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.357681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.421776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafa5ec-2455-441c-ae06-82b8eccbe683 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.426459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d59a2a-eaf0-44d3-90ce-e503a27785f4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.460878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.461534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.461933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.462139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.464678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.465301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.465576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.465846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.466558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:00.466864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.468259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.473290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:00.477030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 544/1090] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:00.478912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:00.479937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:00.481020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 547/1091] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:46:00.483371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.485868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.501209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.602698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00feb697-6e3f-451e-98cb-5b48c3613d96 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.605876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8906ad3-262c-468a-be31-989531b869e9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.613476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.641143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.641762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.642034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.642301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.642997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:00.643295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.643557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.644206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.644483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.644662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.644761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] PATH_INFO: `/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.645968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] RBAC: Authorizing `identity:get_user(user_id=aa4bdffc1c0f4fc1bde7d2a75205d084)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.647440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.649070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 545/1092] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/users/aa4bdffc1c0f4fc1bde7d2a75205d084 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:00.653566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 548/1093] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:46:00.655127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.659352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.730195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.763121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f2e7d-02d0-416a-b8e2-4d1661456853 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.765123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad258a8-630e-45b5-bf29-cc62366d5028 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.797520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.798158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.798439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.798735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.800994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.801594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.801872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.802145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.810295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.811923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:46:00.827343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:00.828396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:46:00.829451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:46:00.829451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:46:00.829451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:00.829451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 549/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:46:00.834666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.854568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.881401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.882818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.938890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56084dcb-88e2-4e49-a6e5-42963592562a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:00.962527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:00.976704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:00.977329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:00.977689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:00.977967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:00.978648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:00.978951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:00.980399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:00.991897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 546/1095] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:00.993217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 550/1096] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:46:00.997948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:00.998305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.080876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:01.110950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7f210-3a6d-48e4-9dbb-909be1bca1b5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.112534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5aefc-89e4-4cb8-bf7c-e409126a7d76 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.149046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.149357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.149735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.149836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.150022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.150160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.150285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.150410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.151451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.152919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.154597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 547/1097] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:01.157958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.159526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.160764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:46:01.172263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:01.173284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:01.174678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 551/1098] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Sep 13 19:46:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:01.180304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.224708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:01.272911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a10d-e4e0-43c0-bba5-b6843c92f184 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.286406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b26ad2-ccbe-4610-b331-02a85d756942 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.307744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.308370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.308644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.308915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.309657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=6dc869d1fe474af88431f3bffae94741, user.id=aa4bdffc1c0f4fc1bde7d2a75205d084) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:01.309956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.311427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.318132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 548/1099] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:01 2022] GET /identity/v3/role_assignments?user.id=aa4bdffc1c0f4fc1bde7d2a75205d084&scope.project.id=6dc869d1fe474af88431f3bffae94741&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:01.322938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.323594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.323866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.324141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.324909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:01.325221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.326768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.349328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 552/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Sep 13 19:46:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:01.355526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.365419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:01.469183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614c0d5-16d2-43e6-bbfb-0200c3fa0986 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.508916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.509564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.509949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.510242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.586559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.588110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.597190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:01.696344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 549/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:01.702297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.807623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:01.811160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2e43-c865-4b74-b1b9-6b952037286f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:01.848966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:01.849542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:01.849822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:01.850088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:01.861343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:01.863220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:46:01.878399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:01.879415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:46:01.880752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:46:01.880752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:46:01.880752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:01.880752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 553/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:01.885683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:01.997345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641245b7-4d0c-4f8d-bdd5-fbf107078021 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:02.005447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:02.034694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:02.035315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:02.035591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:02.035907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:02.036545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:02.036845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:02.038219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:02.050487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 550/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:02.056580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:02.163651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a9836-e14a-48a6-8ef9-e480885f91be None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:02.199094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:02.199740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:02.200027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:02.200301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:02.207330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:02.208790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:46:02.220712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:02.221780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:02.223271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 554/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:02.228356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:02.334153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115c087-6161-4d4e-91d3-e8c7c5d1b4b3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:02.369937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:02.370602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:02.370890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:02.371152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:02.371848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:02.372151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:02.373582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a454e2-4f74-440e-bb28-fd4e032e1496 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:02.386782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 551/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:46:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:02.392959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:02.409479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:02.497917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a343b09-dfda-49ba-a960-c90b2ad2f707 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:02.533866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:02.534517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:02.534796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:02.535048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:02.535775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] RBAC: Adding query filter params (role.id=3e1e2917a95f4a24bbedde5b7a366f6c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=463fcb701698473bad12e8101b9018ce) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:02.536087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:02.537524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f59c00e-61f5-403f-9d3a-7c9fa13337e8 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:02.546249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 555/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:02 2022] GET /identity/v3/role_assignments?user.id=463fcb701698473bad12e8101b9018ce&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=3e1e2917a95f4a24bbedde5b7a366f6c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:03.545765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404ab6-b126-444a-8d7d-17875f6fc0b8 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:03.546535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404ab6-b126-444a-8d7d-17875f6fc0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:03.546684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404ab6-b126-444a-8d7d-17875f6fc0b8 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:03.548721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 552/1107] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:03.556937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603540df-3194-436b-88cb-f6c0756031fb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:03.557381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603540df-3194-436b-88cb-f6c0756031fb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:03.557473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603540df-3194-436b-88cb-f6c0756031fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:03.577751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-603540df-3194-436b-88cb-f6c0756031fb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:03.578998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-603540df-3194-436b-88cb-f6c0756031fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:03.652581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 556/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:03.920212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df404ab6-b126-444a-8d7d-17875f6fc0b8 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:03.948738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:04.026023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df404ab6-b126-444a-8d7d-17875f6fc0b8 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:04.060073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:04.060712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:04.060991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:04.061260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:04.062435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:04.063909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87fb1-c628-4983-bcae-648f19a3a2a7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:04.065508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 553/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:46:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:04.071790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603540df-3194-436b-88cb-f6c0756031fb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:04.080330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:04.175148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603540df-3194-436b-88cb-f6c0756031fb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:04.210448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:04.211058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:04.211310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:04.211644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:04.212304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:04.213750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183e774-b12a-4e24-ade0-8025c1143747 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:04.235705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 557/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:46:04 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:06.415478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d74b-a6c0-4467-93fc-90899369149d None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:06.416035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d74b-a6c0-4467-93fc-90899369149d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:06.416035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4d74b-a6c0-4467-93fc-90899369149d None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:06.417360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 554/1111] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:06.422796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:06.423224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:06.423332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:06.442604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:06.443817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:06.506820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:06.519655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 558/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:06.788759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d74b-a6c0-4467-93fc-90899369149d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:06.887434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:06.895343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4d74b-a6c0-4467-93fc-90899369149d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:06.929000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:06.929630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:06.929907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:06.930187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] PATH_INFO: `/v3/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:06.931604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] RBAC: Authorizing `identity:get_role(role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:06.932991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:06.934349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 555/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:06 2022] GET /identity/v3/roles/f12fcc11a666405fa05a294664d03492 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:06.940663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:07.045120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe3c15-f45d-432c-b576-63391a14e52e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:07.081141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:07.081778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:07.082060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:07.082335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] PATH_INFO: `/v3/users/139135d1dc984d7fae3d8d9914752d4b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:07.096238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] RBAC: Authorizing `identity:get_user(user_id=139135d1dc984d7fae3d8d9914752d4b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:07.097726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:07.099415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 559/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:06 2022] GET /identity/v3/users/139135d1dc984d7fae3d8d9914752d4b => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:07.105512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:07.209725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cc73-d7d1-4d22-80f2-dcec7672583a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:07.246986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:07.247589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:07.247867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:07.248138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:07.277990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:07.317507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:07.318980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:07.422460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 556/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:07.428714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:07.484070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:07.535689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e9670-0c98-4e8d-8612-de78c01df9c3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:07.571508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:07.571508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:07.571838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:07.572114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:07.573294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:07.574775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9ef7b-aa71-4654-9db1-64a81cc34ee0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:07.576434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 560/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:07 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:07.582807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:07.643682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:07.683611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee66790-5905-48d9-ad1e-fa48e8ebbfe3 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:07.718716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:07.719323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:07.719597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:07.719865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:07.720588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=6dc869d1fe474af88431f3bffae94741, user.id=139135d1dc984d7fae3d8d9914752d4b) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:07.720888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:07.722296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ba65-72da-412d-b10c-c8dfd285886c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:07.728838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 557/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:07 2022] GET /identity/v3/role_assignments?user.id=139135d1dc984d7fae3d8d9914752d4b&scope.project.id=6dc869d1fe474af88431f3bffae94741&role.id=f12fcc11a666405fa05a294664d03492 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:09.869948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9d182-977b-447e-b58f-005a209acebc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:09.870540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9d182-977b-447e-b58f-005a209acebc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:09.870540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9d182-977b-447e-b58f-005a209acebc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:09.872000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 561/1118] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:09.878336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:09.878834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:09.878987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:09.898545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:09.899742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:09.980674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 558/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:10.251797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9d182-977b-447e-b58f-005a209acebc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:10.357747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9d182-977b-447e-b58f-005a209acebc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:10.376362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:10.394862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:10.395831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:10.396291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:10.396671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] PATH_INFO: `/v3/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:10.398893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] RBAC: Authorizing `identity:get_role(role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:10.400286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:10.401635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 562/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:10 2022] GET /identity/v3/roles/f12fcc11a666405fa05a294664d03492 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:10.407904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:10.463257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:10.517809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681a23d-d25a-4403-a05a-6081843d8892 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:10.556122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:10.556782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:10.557058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:10.557331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] PATH_INFO: `/v3/users/139135d1dc984d7fae3d8d9914752d4b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:10.558540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=139135d1dc984d7fae3d8d9914752d4b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:10.560103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:10.561726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 559/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:10 2022] GET /identity/v3/users/139135d1dc984d7fae3d8d9914752d4b => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:10.567835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:10.673885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4551818-088b-4933-abd6-a091f1e6806f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:10.710677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:10.711719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:10.712001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:10.712278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:10.713625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:10.715715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac53503-9af3-439a-a52b-8b3ef58edd14 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:10.717434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 563/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:10 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:10.723832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:10.825378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d60141-01c3-4701-849e-974ceb7ce1c4 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:10.852804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:10.866191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:10.866905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:10.867180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:10.867525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741/users/139135d1dc984d7fae3d8d9914752d4b/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:10.869741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] RBAC: Authorizing `identity:create_grant(project_id=6dc869d1fe474af88431f3bffae94741, user_id=139135d1dc984d7fae3d8d9914752d4b, role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:10.871229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddae5e-15a6-4bd2-a066-2a6e0fee666f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:10.885371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 560/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:46:10 2022] PUT /identity/v3/projects/6dc869d1fe474af88431f3bffae94741/users/139135d1dc984d7fae3d8d9914752d4b/roles/f12fcc11a666405fa05a294664d03492 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:13.010880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748663-44b4-4962-b4f6-9b649de4115e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:13.011463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748663-44b4-4962-b4f6-9b649de4115e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:13.011463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748663-44b4-4962-b4f6-9b649de4115e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:13.012762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 564/1124] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:13.018400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:13.018849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:13.018939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:13.038158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:13.039531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:13.106456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 561/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:46:13.373818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748663-44b4-4962-b4f6-9b649de4115e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:13.435732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:13.485724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748663-44b4-4962-b4f6-9b649de4115e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:13.520756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:13.521390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:13.521665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:13.521936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] PATH_INFO: `/v3/roles/f12fcc11a666405fa05a294664d03492` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:13.523387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] RBAC: Authorizing `identity:get_role(role_id=f12fcc11a666405fa05a294664d03492)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:13.524795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:13.526155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 565/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:13 2022] GET /identity/v3/roles/f12fcc11a666405fa05a294664d03492 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:13.532208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:13.633819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8212dca-3ebb-48f6-b303-3e16c473b475 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:13.647432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:13.670575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:13.671211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:13.671488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:13.671759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] PATH_INFO: `/v3/users/139135d1dc984d7fae3d8d9914752d4b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:13.672891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] RBAC: Authorizing `identity:get_user(user_id=139135d1dc984d7fae3d8d9914752d4b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:13.674341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:13.675987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 562/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:13 2022] GET /identity/v3/users/139135d1dc984d7fae3d8d9914752d4b => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:13.681944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:13.792120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4281be-c7e7-4439-aa88-2f6540782c8a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:13.814715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:13.829373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:13.829996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:13.830277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:13.830583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:13.899606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:13.900988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:14.000614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 566/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:14.006858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:14.107927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3091-5c42-415f-a6db-9823618a8b87 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:14.144101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:14.144710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:14.144981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:14.145244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] PATH_INFO: `/v3/projects/6dc869d1fe474af88431f3bffae94741` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:14.146393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] RBAC: Authorizing `identity:get_project(project_id=6dc869d1fe474af88431f3bffae94741)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:14.147814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5359-0925-45c6-8407-694103ee4a68 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:14.149546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 563/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:14 2022] GET /identity/v3/projects/6dc869d1fe474af88431f3bffae94741 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:14.156205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:14.182033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:14.261188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648d4ad-82ec-416a-ae5b-d948fc6bc10f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:14.295572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:14.296179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:14.296454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:14.296722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:14.297440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] RBAC: Adding query filter params (role.id=f12fcc11a666405fa05a294664d03492, scope.project.id=6dc869d1fe474af88431f3bffae94741, user.id=139135d1dc984d7fae3d8d9914752d4b) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:14.297741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:14.299201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352c319-7fbe-4975-81f3-91f6a59a6d1e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:14.305127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 567/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:14 2022] GET /identity/v3/role_assignments?user.id=139135d1dc984d7fae3d8d9914752d4b&scope.project.id=6dc869d1fe474af88431f3bffae94741&role.id=f12fcc11a666405fa05a294664d03492 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:16.500081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda9176-d2e5-48bc-ba14-4b52ac605c8c None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:16.500640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda9176-d2e5-48bc-ba14-4b52ac605c8c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:16.500640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda9176-d2e5-48bc-ba14-4b52ac605c8c None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:16.501956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 564/1131] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:16.507959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:16.508403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:16.508515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:16.527402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:16.528597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:16.600288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 568/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:16.872197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda9176-d2e5-48bc-ba14-4b52ac605c8c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:16.923977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:16.982744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda9176-d2e5-48bc-ba14-4b52ac605c8c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:17.018275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:17.018940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:17.019212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:17.019481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:17.020623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:17.022088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaf48f-1f8c-417f-a5c4-83785e8be825 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:17.023753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 565/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:46:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:17.030008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:17.048959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:17.147200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455dd0b-7c9a-4d4c-b698-4c1e0ab9c6e0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:17.183726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:17.184348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:17.184629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:17.184901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:17.185559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:17.187045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee713f2b-072a-4039-9ccc-100958f8f43f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:17.216166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 569/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:46:17 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:19.399123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca91d0-6e29-4d33-93f2-a3c66f80294f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:19.399702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca91d0-6e29-4d33-93f2-a3c66f80294f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:19.399702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca91d0-6e29-4d33-93f2-a3c66f80294f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:19.401015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 566/1135] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:19.407115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:19.407617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:19.408030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:19.431271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:19.432511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:19.506278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:19.513092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 570/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:19.774069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca91d0-6e29-4d33-93f2-a3c66f80294f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:19.886224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca91d0-6e29-4d33-93f2-a3c66f80294f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:19.904722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:19.922441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:19.923082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:19.923359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:19.923630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:19.924765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:19.926218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d893a9-b278-4617-b223-d0d27c359a18 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:19.927872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 567/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:46:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:19.934345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:20.038987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32155007-e0a0-4445-af4e-b79676fba12b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:20.075662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:20.076302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:20.076563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:20.076847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:20.077507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:20.079008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230bbfe-72ec-4a2e-8c3e-899ae1729aa4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:20.101969 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 571/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:46:19 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:22.292522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607134-7a1c-477c-87e9-f344a4a7b3d2 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:22.293081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607134-7a1c-477c-87e9-f344a4a7b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:22.293081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607134-7a1c-477c-87e9-f344a4a7b3d2 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:22.294437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 568/1139] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:22.300034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:22.300503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:22.300607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:22.320656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:22.321866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:22.373580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:22.394223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 572/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:22.660041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba607134-7a1c-477c-87e9-f344a4a7b3d2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:22.764989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba607134-7a1c-477c-87e9-f344a4a7b3d2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:22.800037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:22.800677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:22.800944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:22.801212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:22.816996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:22.821924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:22.823052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:22.824113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 569/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:22.829283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:22.931456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02712424-bd56-4178-b3a7-dc4e4ce10c4f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:22.966625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:22.967372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:22.967545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:22.967873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:22.968432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:22.968743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:22.970121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:22.979322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 573/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:22.985710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:23.036362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:23.093761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a45541-df76-461d-8df5-0ffd3924e84d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:23.131973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:23.132834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:23.133106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:23.133373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] PATH_INFO: `/v3/users/9033ce4478724b1e88385c6e5e60fe13` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:23.147698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] RBAC: Authorizing `identity:get_user(user_id=9033ce4478724b1e88385c6e5e60fe13)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:23.149304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:23.151036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 570/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:22 2022] GET /identity/v3/users/9033ce4478724b1e88385c6e5e60fe13 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:23.157090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:23.211810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:23.266015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c231715-8190-4d7e-9915-f869117bed7a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:23.299175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:23.299792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:23.300070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:23.300696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:23.372879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:23.374276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:23.425668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:23.480739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 574/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:23.487361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:23.591142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7174-519d-4a7b-8e68-49d01e8b5b84 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:23.593538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:23.628467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:23.629011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:23.629291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:23.629564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] PATH_INFO: `/v3/projects/572f9c93d74f499084a4a23307790b0a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:23.630808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] RBAC: Authorizing `identity:get_project(project_id=572f9c93d74f499084a4a23307790b0a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:23.632234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8ab08-4a12-4338-877f-2b3eb9b2145e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:23.633948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 571/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:23 2022] GET /identity/v3/projects/572f9c93d74f499084a4a23307790b0a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:23.640381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:23.744217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1a558-b256-4b46-8121-24affa4f1910 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:23.779162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:23.779778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:23.780057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:23.780310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:23.781054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=572f9c93d74f499084a4a23307790b0a, user.id=9033ce4478724b1e88385c6e5e60fe13) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:23.781375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:23.782803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa7c78-3f66-471f-a341-b2fa4956edee None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:23.788556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 575/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:23 2022] GET /identity/v3/role_assignments?user.id=9033ce4478724b1e88385c6e5e60fe13&scope.project.id=572f9c93d74f499084a4a23307790b0a&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:25.934116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36046ea-46bd-44d9-a8a5-1102c4f4770d None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:25.934650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36046ea-46bd-44d9-a8a5-1102c4f4770d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:25.934650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36046ea-46bd-44d9-a8a5-1102c4f4770d None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:25.936195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 572/1147] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:25.941900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:25.942399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:25.942494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:25.965515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:25.966805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:25.970085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:26.040091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 576/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:26.305377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36046ea-46bd-44d9-a8a5-1102c4f4770d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:26.416062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36046ea-46bd-44d9-a8a5-1102c4f4770d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:26.450667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:26.451307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:26.451573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:26.451832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:26.463471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:26.470343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:26.471431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:26.472387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 573/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:26.478172 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:26.589221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4da54-c9be-49f2-af64-cd68f1c79424 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:26.628337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:26.628974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:26.629248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:26.629502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:26.630170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:26.630504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:26.631931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:26.641736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 577/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:26.648309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:26.684742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:26.760681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e46c15-365d-40fe-8a78-276821828efc None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:26.797933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:26.798610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:26.798899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:26.799171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] PATH_INFO: `/v3/users/9033ce4478724b1e88385c6e5e60fe13` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:26.800388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=9033ce4478724b1e88385c6e5e60fe13)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:26.801848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:26.803547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 574/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:26 2022] GET /identity/v3/users/9033ce4478724b1e88385c6e5e60fe13 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:26.810064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:26.813231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:26.922396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93030ab9-cbf2-4327-a601-c4fa94eb9c36 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:26.959702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:26.960529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:26.960694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:26.960966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] PATH_INFO: `/v3/projects/572f9c93d74f499084a4a23307790b0a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:26.962155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=572f9c93d74f499084a4a23307790b0a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:26.963657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8a6c6-4061-4ae0-8284-95cb02864dd4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:26.965336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 578/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:26 2022] GET /identity/v3/projects/572f9c93d74f499084a4a23307790b0a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:26.971643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:27.071713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:27.074974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174692f4-e9fc-4714-a0e8-7bcdabca2fa2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:27.109307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:27.109987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:27.110263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:27.110563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] PATH_INFO: `/v3/projects/572f9c93d74f499084a4a23307790b0a/users/9033ce4478724b1e88385c6e5e60fe13/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:27.112899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] RBAC: Authorizing `identity:create_grant(project_id=572f9c93d74f499084a4a23307790b0a, user_id=9033ce4478724b1e88385c6e5e60fe13, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:27.114331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473ad70-e40e-48c7-94a5-fbf54a079bcb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:27.128776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 575/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:46:26 2022] PUT /identity/v3/projects/572f9c93d74f499084a4a23307790b0a/users/9033ce4478724b1e88385c6e5e60fe13/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:29.251365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aec614-def5-40f8-90e9-405a1a541fc4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:29.251951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aec614-def5-40f8-90e9-405a1a541fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:29.251951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aec614-def5-40f8-90e9-405a1a541fc4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:29.253278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 579/1154] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:29.259169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:29.259591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:29.259715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:29.280458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:29.281715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:29.357223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 576/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:29.622943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aec614-def5-40f8-90e9-405a1a541fc4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:29.697082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:29.730289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aec614-def5-40f8-90e9-405a1a541fc4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:29.766179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:29.766851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:29.767127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:29.767400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:29.777639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:29.782520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:29.783568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:29.784428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 580/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:29.789896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:29.882039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:29.892165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49674f3-e825-4529-a1fa-aed9cf3d311d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:29.926326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:29.927083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:29.927516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:29.927878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:29.928550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:29.928915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:29.930305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:29.938590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 577/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:29.944837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:30.048835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca8496-731f-4c17-ae2a-637ae0da94f5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:30.077971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:30.084866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:30.085507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:30.085775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:30.086043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] PATH_INFO: `/v3/users/9033ce4478724b1e88385c6e5e60fe13` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:30.087241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] RBAC: Authorizing `identity:get_user(user_id=9033ce4478724b1e88385c6e5e60fe13)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:30.088669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:30.090260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 581/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:29 2022] GET /identity/v3/users/9033ce4478724b1e88385c6e5e60fe13 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:30.096488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:30.197590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b277fd-020d-41c2-bfbb-589aac4262c0 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:30.231401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:30.232024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:30.232304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:30.232583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:30.255713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:30.303875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:30.305342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:30.408254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 578/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:30.414591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:30.526977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208a070-81d7-4ba2-9033-100538603416 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:30.561818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:30.562492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:30.562751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:30.563023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] PATH_INFO: `/v3/projects/572f9c93d74f499084a4a23307790b0a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:30.564145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] RBAC: Authorizing `identity:get_project(project_id=572f9c93d74f499084a4a23307790b0a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:30.565574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6b5fb-ad3a-45b6-86c6-e776b16caaa3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:30.567548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 582/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:30 2022] GET /identity/v3/projects/572f9c93d74f499084a4a23307790b0a => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:30.573678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:30.626863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:30.682218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc3381-bcb4-435e-823c-c4a3955abddd None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:30.717096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:30.717707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:30.717980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:30.718229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:30.719016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=572f9c93d74f499084a4a23307790b0a, user.id=9033ce4478724b1e88385c6e5e60fe13) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:30.719325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:30.720770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b8470-658c-43d4-9e80-ad768d95b613 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:30.726856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 579/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:30 2022] GET /identity/v3/role_assignments?user.id=9033ce4478724b1e88385c6e5e60fe13&scope.project.id=572f9c93d74f499084a4a23307790b0a&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:32.886779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19794219-1ea9-45d0-b581-85b9d4fdc7f3 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:32.887370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19794219-1ea9-45d0-b581-85b9d4fdc7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:32.887370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19794219-1ea9-45d0-b581-85b9d4fdc7f3 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:32.888574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 583/1162] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:32.894117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:32.894601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:32.894713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:32.913707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:32.914950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:32.985322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 580/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:33.251054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19794219-1ea9-45d0-b581-85b9d4fdc7f3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:33.322236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:33.367646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19794219-1ea9-45d0-b581-85b9d4fdc7f3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:33.406988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:33.407607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:33.407887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:33.408153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:33.477000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:33.478548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:33.548687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:33.587813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 584/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:33.593989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:33.643408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:33.698237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb814336-d383-4ca3-bb4e-05d0e031df07 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:33.733216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:33.733846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:33.734125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:33.734477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:33.741483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:33.742980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-64fa2474-6666-4771-b45d-a3b29ce63f70 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:33.754929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:33.756169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:33.757300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 581/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:46:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:33.762556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:33.864097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05cd14-61ec-45c9-b2f8-8a02c563d03f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:33.898711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:33.899326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:33.899585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:33.899859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:33.900499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:33.900799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:33.902180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17003-3899-45f5-af92-5da2d8362a19 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:33.909587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 585/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Sep 13 19:46:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:36.050394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d4df-a973-41e8-b760-3f96b90f1eb6 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:36.050993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d4df-a973-41e8-b760-3f96b90f1eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:36.050993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d4df-a973-41e8-b760-3f96b90f1eb6 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:36.052304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 582/1167] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:36.058354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c293f4b-171e-4ad3-bed0-5e1fc3471df5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:36.058840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c293f4b-171e-4ad3-bed0-5e1fc3471df5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:36.059308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c293f4b-171e-4ad3-bed0-5e1fc3471df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:36.078255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-1c293f4b-171e-4ad3-bed0-5e1fc3471df5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:36.079490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1c293f4b-171e-4ad3-bed0-5e1fc3471df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:36.105014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:36.149684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 586/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:36.415064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d4df-a973-41e8-b760-3f96b90f1eb6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:36.528859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d4df-a973-41e8-b760-3f96b90f1eb6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:36.536434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:36.564736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:36.565347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:36.565627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:36.565893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:36.566590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:36.567976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f39ad-0c57-4664-8444-2103a9246d52 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:36.584819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 583/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Tue Sep 13 19:46:36 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:38.767123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaed5a-1112-43d6-800b-f1d6030625d1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.767654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaed5a-1112-43d6-800b-f1d6030625d1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.767654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaed5a-1112-43d6-800b-f1d6030625d1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.769041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 587/1170] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:38.775537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.775979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.776087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.795457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:38.796741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:38.873939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 584/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:39.136409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaed5a-1112-43d6-800b-f1d6030625d1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:39.255200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaed5a-1112-43d6-800b-f1d6030625d1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:39.293864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:39.294546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.294830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.295106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] PATH_INFO: `/v3/domains/f6e83192923e40c2b8cc91bbcc3790d6` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.306405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f6e83192923e40c2b8cc91bbcc3790d6)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:39.307885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddbd37-003c-460e-8867-dab5adf56210 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:39.309549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 588/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Sep 13 19:46:39 2022] GET /identity/v3/domains/f6e83192923e40c2b8cc91bbcc3790d6 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:39.318061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:39.416210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:39.423407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66056bd3-6084-4945-bf0f-518576feefb5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:39.465945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:39.466673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.466947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.467216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.467867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:39.469309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ef609-7235-4e3e-9265-09d76e62573d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:39.499452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 585/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:46:39 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:41.716511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921a9f-61b7-4e06-b18d-86f80d87a6c0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:41.717041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921a9f-61b7-4e06-b18d-86f80d87a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:41.717041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60921a9f-61b7-4e06-b18d-86f80d87a6c0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:41.718425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 589/1174] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:41.724086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae866-2c12-4474-a889-c14e663091db None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:41.724543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae866-2c12-4474-a889-c14e663091db None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:41.724636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabae866-2c12-4474-a889-c14e663091db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:41.744559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-cabae866-2c12-4474-a889-c14e663091db None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:41.745785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cabae866-2c12-4474-a889-c14e663091db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:41.815195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 586/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:46:42.085063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60921a9f-61b7-4e06-b18d-86f80d87a6c0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:42.108989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:42.194648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60921a9f-61b7-4e06-b18d-86f80d87a6c0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:42.229444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:42.230078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:42.230353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:42.230653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] PATH_INFO: `/v3/domains/f6e83192923e40c2b8cc91bbcc3790d6` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:42.231770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=f6e83192923e40c2b8cc91bbcc3790d6)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:42.233226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46dadd-a676-417b-b8bd-4665b25eba3b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:42.234857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 590/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Sep 13 19:46:42 2022] GET /identity/v3/domains/f6e83192923e40c2b8cc91bbcc3790d6 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:42.241267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae866-2c12-4474-a889-c14e663091db None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:42.321432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:42.345319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabae866-2c12-4474-a889-c14e663091db None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:42.380084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:42.380706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:42.380983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:42.381251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:42.381901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:42.383377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e5322-06c8-4fba-8d7b-2b86e62f2f90 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:42.406358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 587/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:46:42 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:44.613284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870b897-7967-4f9a-b35b-f82baa83320c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.613887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870b897-7967-4f9a-b35b-f82baa83320c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.613887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870b897-7967-4f9a-b35b-f82baa83320c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.615361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 591/1178] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:44.621503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.621972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.622096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.640896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:44.642116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:44.712617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 588/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:44.978157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870b897-7967-4f9a-b35b-f82baa83320c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.070912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.085674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870b897-7967-4f9a-b35b-f82baa83320c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.121506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.122138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.122418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.122717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.133245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:45.138096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:45.139301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:45.140205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 592/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:45.145512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.209757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.249634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2129a-1cc2-4148-bc49-0e86fe3c8a14 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.284930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.285537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.285816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.286082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.286775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:45.287135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.288507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.296904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 589/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:45.303352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.410044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df802e-b1e5-4c9c-9d5a-d7eb908d21ba None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.445145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.445773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.446064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.446338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] PATH_INFO: `/v3/users/ba940c7857624c38ab1de6b3adfdcdd2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.460037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] RBAC: Authorizing `identity:get_user(user_id=ba940c7857624c38ab1de6b3adfdcdd2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.461500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.463197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 593/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/users/ba940c7857624c38ab1de6b3adfdcdd2 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:45.470298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.575367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9816e7-330a-4896-b1c9-38799a7158f5 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.602967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.612834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.613455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.613729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.613999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.681944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.683430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.784622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 590/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:45.791216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.800782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.895058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7f597-8b63-4023-a123-f6da4b87d161 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.930425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.931088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.931368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.931636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] PATH_INFO: `/v3/projects/68334b8857b04782ae2136773324a370` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.932856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] RBAC: Authorizing `identity:get_project(project_id=68334b8857b04782ae2136773324a370)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.934299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97d176-4526-4257-8c41-07e9bb7fe660 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.936006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 594/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/projects/68334b8857b04782ae2136773324a370 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:45.942702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.972574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.055352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ea842-a925-4054-b4ba-1d96980acd42 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:46.091401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:46.092007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.092280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.092547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.093271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=68334b8857b04782ae2136773324a370, user.id=ba940c7857624c38ab1de6b3adfdcdd2) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:46.093570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:46.095036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b5da-caf6-40dc-bc4a-d09f0d3344d7 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:46.100719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 591/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/role_assignments?user.id=ba940c7857624c38ab1de6b3adfdcdd2&scope.project.id=68334b8857b04782ae2136773324a370&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:48.240550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b800cb-2b2c-4b22-99a7-11e8c97b0569 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:48.241089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b800cb-2b2c-4b22-99a7-11e8c97b0569 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:48.241089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b800cb-2b2c-4b22-99a7-11e8c97b0569 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:48.242511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 595/1186] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:48.250570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:48.251261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:48.251518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:48.272077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:48.273280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:48.352649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 592/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:46:48.648277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b800cb-2b2c-4b22-99a7-11e8c97b0569 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:48.733019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:48.762763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b800cb-2b2c-4b22-99a7-11e8c97b0569 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:48.804056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:48.804694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:48.804977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:48.805253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:48.816207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:48.821404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:48.822467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:46:48.823325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 596/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:48.828991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:48.853294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:48.932467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06922c15-5637-4850-baa0-1155b02ea056 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:48.967034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:48.967647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:48.967925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:48.968199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:48.968888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:48.969186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:48.970595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:48.979217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 593/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:48.985767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:49.091461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf01c6-7751-4286-9cc9-8a603799f803 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:49.130343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:49.130999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:49.131275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:49.131541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] PATH_INFO: `/v3/users/ba940c7857624c38ab1de6b3adfdcdd2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:49.132723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] RBAC: Authorizing `identity:get_user(user_id=ba940c7857624c38ab1de6b3adfdcdd2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:49.134159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:49.135781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 597/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:48 2022] GET /identity/v3/users/ba940c7857624c38ab1de6b3adfdcdd2 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:49.141921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:49.226868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:49.246442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b6c85-c1ad-40d4-8a80-ec51c243f33a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:49.281596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:49.282303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:49.282510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:49.282776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] PATH_INFO: `/v3/projects/68334b8857b04782ae2136773324a370` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:49.283900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] RBAC: Authorizing `identity:get_project(project_id=68334b8857b04782ae2136773324a370)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:49.285441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e47ec-e7f7-42d6-9810-36e7b4780a8a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:49.287120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 594/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:49 2022] GET /identity/v3/projects/68334b8857b04782ae2136773324a370 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:49.293307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:49.296456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:49.400557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b40940-aa88-4029-bf83-fd42f95b7366 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:49.435778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:49.436486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:49.436767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:49.437039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] PATH_INFO: `/v3/projects/68334b8857b04782ae2136773324a370/users/ba940c7857624c38ab1de6b3adfdcdd2/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:49.439072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=68334b8857b04782ae2136773324a370, user_id=ba940c7857624c38ab1de6b3adfdcdd2, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:49.440527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7683dd7-35a2-4cfc-b5a2-71fc64ffddc3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:49.455391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 598/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:46:49 2022] PUT /identity/v3/projects/68334b8857b04782ae2136773324a370/users/ba940c7857624c38ab1de6b3adfdcdd2/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:51.573858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f5b8-204b-4cba-a150-4b10c0145751 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:51.574354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f5b8-204b-4cba-a150-4b10c0145751 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:51.574496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f5b8-204b-4cba-a150-4b10c0145751 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:51.575834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 595/1193] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:51.581772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:51.582347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:51.582347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:51.601421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:51.602658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:51.663704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:51.675592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 599/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:46:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:46:51.940281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f5b8-204b-4cba-a150-4b10c0145751 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:52.047881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f5b8-204b-4cba-a150-4b10c0145751 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:52.086514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:52.087204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:52.087456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:52.087737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:52.101343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:46:52.106304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:46:52.107533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:46:52.108666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 596/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:46:52.114897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:52.224229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71048c59-9b10-4b56-8395-1a94abbc14df None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:52.260347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:52.260967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:52.261251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:52.261514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:52.262187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:52.262523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:52.263897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:52.272359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 600/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:52.278844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:52.293048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:52.393818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43b57c-09f2-43d5-8170-737eb4c47de8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:52.431387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:52.432004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:52.432278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:52.432573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] PATH_INFO: `/v3/users/ba940c7857624c38ab1de6b3adfdcdd2` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:52.433734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] RBAC: Authorizing `identity:get_user(user_id=ba940c7857624c38ab1de6b3adfdcdd2)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:52.435179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:52.436831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 597/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Sep 13 19:46:52 2022] GET /identity/v3/users/ba940c7857624c38ab1de6b3adfdcdd2 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:52.443035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:52.502225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:52.549068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b403a85-7276-4b8b-8a08-f67248249a7a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:52.584181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:52.584872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:52.585074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:52.585401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:52.655530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:52.657019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:52.714261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:52.761585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 601/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:46:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:46:52.767666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:52.857134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:52.883487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87440882-f2d1-4cb9-87b8-837b926284f6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:52.918553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:52.919180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:52.919451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:52.919723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] PATH_INFO: `/v3/projects/68334b8857b04782ae2136773324a370` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:52.920901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] RBAC: Authorizing `identity:get_project(project_id=68334b8857b04782ae2136773324a370)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:52.922337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9c55d-458f-4a1c-8c98-9c455c646332 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:52.924058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 598/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Sep 13 19:46:52 2022] GET /identity/v3/projects/68334b8857b04782ae2136773324a370 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:52.931264 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:53.046294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7fc34-9ee6-419e-a1cb-a8f68310e368 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:53.083449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:53.084071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:53.084357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:53.084616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:53.085360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=68334b8857b04782ae2136773324a370, user.id=ba940c7857624c38ab1de6b3adfdcdd2) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:53.085636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:53.087218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf18a2-caf8-41f8-87fd-fcb04ff877bb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:53.093553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 602/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:46:52 2022] GET /identity/v3/role_assignments?user.id=ba940c7857624c38ab1de6b3adfdcdd2&scope.project.id=68334b8857b04782ae2136773324a370&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:47:02.913303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=27 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:03.090728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:50.173377 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f71c0-bf84-4735-bd78-bd7bf51b4a21 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:50.173968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f71c0-bf84-4735-bd78-bd7bf51b4a21 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:50.173968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f71c0-bf84-4735-bd78-bd7bf51b4a21 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:50.175559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 599/1201] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:50.184550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:50.184550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:50.184550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:50.206489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:50.207776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:50.305275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 603/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:47:50.594654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f71c0-bf84-4735-bd78-bd7bf51b4a21 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:50.740100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f71c0-bf84-4735-bd78-bd7bf51b4a21 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:50.782447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:50.782810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:50.783242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:50.783670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:50.795933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:50.797412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:47:50.813134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:50.814405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:47:50.815552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:47:50.815552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:50.815552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:47:50.815552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 600/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:47:50.820832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:50.965953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035069c-b701-41ad-9fa4-32156be4c0a7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:51.020703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:51.037884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:51.038931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:51.039416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:51.039890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:51.040948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:51.041459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:51.043496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61610c53-5103-4ee3-93b0-1b5d6bed1131 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:51.064208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 604/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:47:51.073444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:51.178127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:51.262417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3e342-6e96-4476-b593-7da7b231db5b None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:51.328112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:51.329100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:51.329733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:51.329927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:51.330627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:51.332786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86e31f-1c16-420e-96a0-a56456e2e54f None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:51.358102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 601/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Sep 13 19:47:51 2022] POST /identity/v3/users => generated 251 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:47:54.106620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10630-99c7-49df-8464-1ba2ff6c246d None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:54.109041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10630-99c7-49df-8464-1ba2ff6c246d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:54.109041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff10630-99c7-49df-8464-1ba2ff6c246d None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:54.109368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 605/1206] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:47:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:54.116950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:54.117491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:54.117491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:54.136741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:54.137972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:54.214190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 602/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:47:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:47:54.485206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff10630-99c7-49df-8464-1ba2ff6c246d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:54.608918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff10630-99c7-49df-8464-1ba2ff6c246d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:54.653406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:54.654092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:54.654386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:54.654668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:54.667499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:47:54.673141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:54.674165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:47:54.675133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 606/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:47:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:47:54.680869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:54.728260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:54.803837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c2a83d-9b75-47d7-b8a8-15dcbdf9bbcd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:54.841576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:54.842203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:54.842513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:54.842784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:54.843452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:54.843755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:54.845202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:54.854522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 603/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:47:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:47:54.860883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:54.927158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:54.994863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a86cf-53d3-4ad5-803f-155677d27bb0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:55.032283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:55.033311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:55.033890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:55.034426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:55.047473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:55.049307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:47:55.068393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:55.069740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:47:55.070917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:47:55.070917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:55.070917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:47:55.074456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 607/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:47:55.079042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:55.193854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:55.197456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae8040-0871-42e2-8c51-4d0ed057d8f6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:55.240158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:55.240786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:55.241067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:55.241342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:55.241996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:55.242301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:55.243735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:55.256264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 604/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:47:55.262769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:55.383210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfc030-7654-4bdb-adfb-735f506caa6f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:55.385208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:55.433809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:55.434488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:55.434773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:55.435016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:55.443353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:55.445084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:47:55.458591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:55.459670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:47:55.460653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 608/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:47:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:47:55.465995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:55.588864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46493b8-e3f8-4434-aa0a-565e5f936ab9 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:55.631387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:55.632022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:55.632300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:55.632569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:55.633477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:55.633621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:55.635324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:55.658417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 605/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:47:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:47:55.667634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:55.794409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678723e-5522-4fa9-85cf-fc1d34f66dbe None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:55.833417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:55.834036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:55.834323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:55.834611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:55.868834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:55.910436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:55.911811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.030903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 609/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:47:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:47:56.032603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:56.055167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:56.153427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308dc5e-2fe5-42fb-ada4-32589319e9ee None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:56.197139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:56.197982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:56.198137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:56.198495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:56.212369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:56.214085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:47:56.229393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:56.231546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:47:56.232750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:47:56.232750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:47:56.232750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:47:56.232750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 606/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:47:56.237101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:56.302216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:56.346506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f645d43-c0bf-4942-80cf-5f5747cc9d7f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:56.388663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:56.389444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:56.389813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:56.390139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:56.391017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:56.391429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:56.393308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.407029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 610/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:47:56.413556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:56.501170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:56.542413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bff8c-caa5-43fe-82f0-c468fcd02c37 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:56.581556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:56.582196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:56.582500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:56.582771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:56.589583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:56.591093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:47:56.603506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:56.604980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:47:56.606129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:47:56.610088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 607/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:47:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:47:56.611389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:56.737669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6200c-0b08-450c-b26d-1b086e79b37e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:56.760434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:56.774057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:56.774488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:56.774863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:56.775206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:56.776186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:56.776691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:56.778796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340673b2-682d-4fc9-a60d-442f01a20ca2 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.798097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 611/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:47:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:47:56.804546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:56.914573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0063a41-ef6c-4236-97fa-957cf8dce647 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:56.945460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:56.951514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:56.953169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:56.953169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:56.953169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:56.953471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=e7bec1fe5f26425eb9d3e98f7502a7d0) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:47:56.953688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:56.955167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998070c-9b4f-4723-99fb-b37e4448af15 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:56.961840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 608/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:47:56 2022] GET /identity/v3/role_assignments?user.id=e7bec1fe5f26425eb9d3e98f7502a7d0&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:47:59.356999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880c54-8939-426d-ac0a-01bd2f0e28ec None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:59.357597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880c54-8939-426d-ac0a-01bd2f0e28ec None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:59.357597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36880c54-8939-426d-ac0a-01bd2f0e28ec None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:59.359050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 612/1220] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:47:59.365195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:59.365693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:59.365795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:59.385382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:59.386749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:59.458066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 609/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:47:59.761584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36880c54-8939-426d-ac0a-01bd2f0e28ec None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:59.873535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36880c54-8939-426d-ac0a-01bd2f0e28ec None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:59.910546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:59.911179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:59.911454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:59.911720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:59.922137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:47:59.927161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:47:59.928368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:47:59.929198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 613/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:47:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:47:59.934824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.046916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2fc5a-bcc7-43a8-89c0-1a9bf685a39f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:00.074616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:00.088901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:00.089529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:00.089842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:00.090140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:00.091003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:00.091311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:00.093510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:00.104232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 610/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:47:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:00.110706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.161296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:00.223678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08cd7c-00d9-4879-a15a-2af3a344199b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:00.267243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:00.268243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:00.268450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:00.268723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:00.283373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:00.285105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:00.301620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:00.303072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:00.304208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:00.304208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:00.304208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:00.304208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 614/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:00.309664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.424891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4693d-7dba-4406-963b-7f9bd80e7a60 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:00.467971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:00.468595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:00.468883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:00.469177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:00.469780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:00.470087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:00.471494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:00.483929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 611/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:00.490278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.590183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:00.597208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63abe7c-1725-4a2f-8a56-4e49309f1d5e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:00.637772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:00.638430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:00.638725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:00.639024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:00.648301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:00.649902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:48:00.662840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:00.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:00.665620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 615/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:48:00.670433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.726491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:00.787881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8e9e0-7457-429f-bceb-0121f141af3c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:00.824443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:00.824981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:00.825259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:00.825571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:00.826331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:00.826726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:00.828301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:00.849895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 612/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:48:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:48:00.857031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:00.968449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601589ea-a865-4890-b85f-a48eb5661473 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:01.003625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:01.004312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:01.004638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:01.004907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:01.016673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:01.018190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:01.033218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:01.034321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:01.035403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:01.035403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:01.035403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:01.035403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 616/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:01.042740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:01.151058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76bc16-9f86-43dc-80b6-da6c40b51440 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:01.153510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:01.188244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:01.188866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:01.189146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:01.189419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:01.190121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:01.190470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:01.191922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:01.205286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 613/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:01.211601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:01.221913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:01.323109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738a5a-ab7e-4acc-b841-0d24b1b0a0c5 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:01.361801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:01.362529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:01.362810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:01.363080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:01.370564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:01.372064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:48:01.384534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:01.385620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:01.387131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 617/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:01.391794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:01.497861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce5403-8788-4cfc-bc47-36c759205e0e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:01.534234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:01.535039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:01.535384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:01.535718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:01.536489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:01.536795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:01.538289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5eac22-0a20-454a-b47e-0a9bb8702ada None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:01.550796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 614/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:01.556981 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:01.625098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:01.664738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c8a6c-5032-4aa3-8d2d-0b63b2ca7ea7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:01.701971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:01.702669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:01.702968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:01.703273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/e7bec1fe5f26425eb9d3e98f7502a7d0/roles/b0f478d9ee7f4e7389a501bfc83fff03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:01.721437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=e7bec1fe5f26425eb9d3e98f7502a7d0, role_id=b0f478d9ee7f4e7389a501bfc83fff03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:01.723073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da97af-2636-4f6d-afbd-e22031392330 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:01.738880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 618/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:48:01 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/e7bec1fe5f26425eb9d3e98f7502a7d0/roles/b0f478d9ee7f4e7389a501bfc83fff03 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:48:03.982476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece493-57f3-4654-81ff-b0f7a907c101 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:03.983032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece493-57f3-4654-81ff-b0f7a907c101 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:03.983032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ece493-57f3-4654-81ff-b0f7a907c101 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:03.984509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 615/1233] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:03.992015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:03.992467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:03.992979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:04.014955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:04.016249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:04.101536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 619/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:04.387206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece493-57f3-4654-81ff-b0f7a907c101 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:04.415489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:04.505297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ece493-57f3-4654-81ff-b0f7a907c101 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:04.542515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:04.543267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:04.543483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:04.543720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] PATH_INFO: `/v3/roles/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:04.556938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:48:04.562052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:04.563291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:04.564451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 616/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Sep 13 19:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:04.572710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:04.636034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:04.698721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d7711-dfd8-4456-add7-9d003b7da7f2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:04.742073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:04.742952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:04.743234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:04.743650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:04.744386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] RBAC: Adding query filter params (name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:04.744717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:04.746391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:04.757153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 620/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Sep 13 19:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:04.763276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:04.855738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:04.873431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8ed5-84f5-4410-b733-788c5e492def None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:04.910331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:04.910918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:04.911220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:04.911517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:04.924403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:04.926012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:04.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:04.943357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:04.944730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:04.944730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:04.944730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:04.944730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 617/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:04.950245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:05.067725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763627b-17bd-44ea-b92d-8fe07461482e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:05.077872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:05.106654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:05.107280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:05.107555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:05.107822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:05.108467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:05.108766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:05.110160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:05.122771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 621/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:48:05.128963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:05.243655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6192f90-8d6d-4b51-90e2-4699af663005 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:05.280126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:05.280771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:05.281048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:05.281317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:05.284042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:05.289625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:05.291299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:48:05.302417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:05.303615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:05.304734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 618/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:05.309761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:05.430640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507ee6a-3a7e-4991-8159-202bfae7996b None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:05.468277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:05.468896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:05.469174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:05.469443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:05.470176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:05.470506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:05.472001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:05.490044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 622/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:48:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:05.496637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:05.619013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d73f-9383-44e1-b817-eff1e2a2a3d0 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:05.657607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:05.658273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:05.658635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:05.658913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:05.735487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:05.745877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:05.747817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:05.862983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 619/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:48:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:48:05.869310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:05.899640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:05.980947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44089e8-86ad-4840-9d54-246ade0cb6d3 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:06.019448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:06.020010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:06.020288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:06.020549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:06.031325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:06.032783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:06.048368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:06.049795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:06.051088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:06.051088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:06.051088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:06.051088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 623/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:06.055994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:06.165592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:06.172329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc78637-2e9e-40cc-a936-07c27fae227c None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:06.210127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:06.210797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:06.211074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:06.211406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:06.212098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:06.212407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:06.213851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:06.226201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 620/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:06.232974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:06.319205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:06.344378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ceda3-830e-4e84-a04f-481dcf3d6be4 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:06.383874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:06.384888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:06.385170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:06.385437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:06.395185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:06.395185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:48:06.409574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:06.410592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:06.416877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 624/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:06.417534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:06.539260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6e265-a8c6-45d9-bed0-6ddb5c2964ff None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:06.585120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:06.585743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:06.586021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:06.586286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:06.587047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:06.587309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:06.589249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e991053-bb40-453b-ac40-7cce9b354339 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:06.605093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 621/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:06.612656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:06.743055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddc36c-334b-4333-aff9-7b4d5920c1c9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:06.786569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:06.787201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:06.788061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:06.788061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:06.788471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] RBAC: Adding query filter params (role.id=b0f478d9ee7f4e7389a501bfc83fff03, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=e7bec1fe5f26425eb9d3e98f7502a7d0) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:06.788785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:06.790226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029e1b4-9449-43fb-92a7-21d854b84efb None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:06.796667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 625/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:48:06 2022] GET /identity/v3/role_assignments?user.id=e7bec1fe5f26425eb9d3e98f7502a7d0&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=b0f478d9ee7f4e7389a501bfc83fff03 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:09.314160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee76f-1057-4786-af16-2a838e02df09 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:09.314784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee76f-1057-4786-af16-2a838e02df09 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:09.314784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ee76f-1057-4786-af16-2a838e02df09 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:09.316302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 622/1247] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:09.322182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:09.322702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:09.322826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:09.329468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:09.344793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:09.346383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:09.424429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 626/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:09.685885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ee76f-1057-4786-af16-2a838e02df09 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:09.704512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:09.809611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ee76f-1057-4786-af16-2a838e02df09 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:09.848991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:09.849669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:09.849937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:09.850311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:09.865830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:48:09.870907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:09.871947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:09.872766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 623/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:09.878251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:09.987857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ae7e-4db4-4a8a-a622-759981735e05 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.019439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:10.025121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.025890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.026266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.026627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.027489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:10.027884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:10.029629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:10.039990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 627/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:10.046080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:10.130449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:10.157000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19aa2-e79d-4753-b1a9-839c23029f84 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.194011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.194681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.194956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.195230 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.206406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:10.208073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:10.223109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:10.224247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:10.225923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:10.225923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:10.225923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:10.225923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 624/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:48:10.230836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:10.349747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924cdbd8-a04b-45f9-a857-2b6a0d447363 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.389800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.390577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.390992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.391474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.392507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:10.393099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:10.395441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:10.418004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 628/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:10.419569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:10.536309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeb437-db7f-4c58-b3fc-5d2c2311c863 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.568999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:10.577790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.578753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.579240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.579547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.589606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:10.591227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:48:10.606130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:10.607456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:10.608614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 625/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:10.614430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:10.683050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:10.742333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca24ea2-3fbb-4daa-b94d-68191630319d None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.793263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.793902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.794186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.794508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.795260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:10.795567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:10.797041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:10.817195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 629/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:48:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:10.823313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:10.950330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b55e9c-ea5f-4390-a521-98c37f39563d None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:10.990947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:10.991639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.991921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.992190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.058408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:11.072949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.074544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.187792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 626/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:48:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:48:11.194624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.306311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c3e02-84cd-474e-a289-0d3ea5673759 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:11.320731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:11.343591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:11.344237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:11.344494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:11.344770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.355713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.357176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:11.371683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:11.372728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:11.373780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:11.373780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:11.373780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:11.373780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 630/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:48:11.379156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.467148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:11.480643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f56773-5c98-4aca-8f76-6157d02d2498 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:11.521234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:11.521843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:11.522116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:11.522414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.523048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:11.523352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.524752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.536287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 627/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:11.542413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.647556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24685f-82a8-4a3c-b801-b95ae885b5b0 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:11.682662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:11.683295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:11.683572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:11.683912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.690704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.692189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:48:11.703627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:11.704995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:11.705980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 631/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:48:11.711550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.823263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45f1b3-6f64-4989-979c-7e203cde493e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:11.859795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:11.860417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:11.860692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:11.860960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.861727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:11.862029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.863491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538842e9-488b-404a-b46e-f7fefbc51c57 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.875946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 628/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:48:11.882671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.911517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:11.994413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481d29-2434-4f6d-8858-9c63e174317f None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:12.029383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:12.029986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.030262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.030647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.031366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=e7bec1fe5f26425eb9d3e98f7502a7d0) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:12.031666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:12.033152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d63e29-d443-43de-98aa-732a9db2a652 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:12.039390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 632/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/role_assignments?user.id=e7bec1fe5f26425eb9d3e98f7502a7d0&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:14.194916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff535a-1cd5-4b0b-8334-5128b614094a None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:14.195467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff535a-1cd5-4b0b-8334-5128b614094a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:14.195467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff535a-1cd5-4b0b-8334-5128b614094a None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:14.197536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 629/1261] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:14.205430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:14.205895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:14.205992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:14.225912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:14.227134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:14.295478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 633/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:48:14.560582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff535a-1cd5-4b0b-8334-5128b614094a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:14.573221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:14.671753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff535a-1cd5-4b0b-8334-5128b614094a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:14.714721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:14.715360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:14.715657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:14.715885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:14.726583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:48:14.731971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:14.734033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:14.735092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 630/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:48:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:14.741187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:14.762254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:14.845275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382eb0e4-50ac-455e-973e-9992fb964077 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:14.882604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:14.883631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:14.884109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:14.884536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:14.885517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:14.886123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:14.887635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:14.896895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 634/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:48:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:14.903251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:14.981169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:15.016395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ef95-cb7c-40b8-900e-89ad647f5574 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.051576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.052198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.052472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.052735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.063394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.064877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:15.078908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:15.080034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:15.081253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:15.081253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:15.081253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:15.081253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 631/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:15.086183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:15.172191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:15.198933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fe913-1a71-44bb-9f6c-7527faaa1e69 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.245991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.246747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.246872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.247175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.247815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:15.248114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.249489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.262940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 635/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:48:15.272595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:15.383934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4dfc1-628f-4b8a-a237-bac2768a20a8 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.404022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:15.419713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.420342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.420620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.420895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.428693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.430079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:48:15.441331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:15.442467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:15.443888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 632/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:15.449068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:15.555648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb9a0f-bb33-454a-b5c6-097f53f642d9 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.593446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.594073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.594356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.594691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.595454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:15.595759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.597212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.606421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:15.615526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 636/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:15.624730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:15.743614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8e8e9-9399-47ff-a39f-0d6f137c4a6a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.777975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.778630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.778902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.779167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.789937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.791420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:15.805742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:15.806863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:15.807899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:15.807899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:15.807899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:15.810832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 633/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:15.813332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:15.929797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9437d0e-d6fb-48d3-a9d1-9731bfa8cbf7 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:15.967817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:15.968432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:15.968714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:15.968982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:15.969626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:15.969927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:15.971492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:15.983607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 637/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:15.990229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:16.026579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:16.103564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d59a-e378-444f-a052-2e353f16ade2 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:16.139156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:16.139799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:16.140077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:16.140349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:16.147493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:16.148955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:48:16.161218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:16.162278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:16.163537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 634/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:16.168549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:16.229337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:16.274440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bb135-047d-437c-bf27-79982f84ca3e None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:16.308058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:16.308591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:16.308881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:16.309343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:16.309842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:16.310154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:16.311647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75f67b-8898-401c-9fa8-153e97823a56 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:16.323650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 638/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:16.329938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:16.442065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:16.452191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667e76d-0e04-4b10-a282-8faae699e248 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:16.490928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:16.491586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:16.491863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:16.492131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] PATH_INFO: `/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/e7bec1fe5f26425eb9d3e98f7502a7d0/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:16.494157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] RBAC: Authorizing `identity:create_grant(project_id=5597c130581c48e7bcd2fdd79c5e72e0, user_id=e7bec1fe5f26425eb9d3e98f7502a7d0, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:16.495642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9e053-6582-44b4-a84c-b047079d8576 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:16.511324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 635/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Sep 13 19:48:16 2022] PUT /identity/v3/projects/5597c130581c48e7bcd2fdd79c5e72e0/users/e7bec1fe5f26425eb9d3e98f7502a7d0/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:18.634245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d330a-dcf4-49cc-ac3f-9b176b5280c1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:18.634861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d330a-dcf4-49cc-ac3f-9b176b5280c1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:18.634861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d330a-dcf4-49cc-ac3f-9b176b5280c1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:18.636199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 639/1274] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:18.642695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:18.643380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:18.643556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:18.668290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:18.669498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:18.750979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 636/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:19.015839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d330a-dcf4-49cc-ac3f-9b176b5280c1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.122613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d330a-dcf4-49cc-ac3f-9b176b5280c1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:19.141259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:19.159639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:19.160254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.160529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.160795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.171951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:48:19.176879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:19.177879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:19.178904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 640/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:19.184448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.268543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:19.288553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aac13-f691-4bf4-b0e1-fef060735cf5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:19.327480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:19.328091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.328367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.328635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.329312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:19.329674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:19.331067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:19.339196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 637/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:19.345657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.455669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094e219-b230-4195-bc32-f93f91f16c55 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:19.494288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:19.494957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.495241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.495486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.506637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:19.508144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:19.524420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:19.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:19.527928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:19.527928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:19.527928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:19.527928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 641/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:19.534284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.648457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ea08b-1313-4d17-93d6-b5a0661a7f9e None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:19.685494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:19.686099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.686426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.686678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.687327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:19.687631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:19.689044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:19.700764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 638/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:19.707140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.741743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:19.813916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff796a-d363-4e0a-a7ae-edd9e21e113a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:19.850828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:19.851710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.852070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.852398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.863219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:19.864667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:48:19.876272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:19.877484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:19.878578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 642/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:19.883649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:19.887979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:19.989832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fee01-fb4c-4e71-be72-7c34a7cacb52 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.025957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.026599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.026871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.027134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.027858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:20.028142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.029568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.049336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 639/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Sep 13 19:48:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:20.055458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.148472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:20.158142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859bce-ea8f-4758-b720-e9a44e1e13fa None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.193210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.193832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.194113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.194424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.264625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.266093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.366951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:20.379683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 643/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:48:20.385891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.449258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:20.497277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d045a-ff22-4fd3-9a91-0550511b4d36 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.533330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.533996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.534272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.534587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.545951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.547436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:48:20.562164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:20.563282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:48:20.564198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:48:20.564198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:48:20.564198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:20.564198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 640/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:20.569577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.681170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11d731-ef65-44a1-b552-e38033777173 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.719134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.719817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.720012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.720277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.720919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:20.721216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.722615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.735015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 644/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:20.741578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.848910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf359fdb-9769-482b-be36-92afe85b5768 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.862863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:20.887863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.888593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.888879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.889159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.897279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.898740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:48:20.911063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:20.912066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 19:48:20.913128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 641/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:20.918598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.980402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:21.045671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14532bc-8efe-4e9d-b6eb-5848ac580b0a None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:21.084671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:21.085284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:21.085562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:21.085817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:21.086651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:21.086959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:21.088418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c648c-7317-4420-98f3-49db94431512 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:21.103756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 645/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:21.111321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:21.236680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852e03c-76e5-4353-8369-7bde6ec8eb6a None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:21.270866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:21.271480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:21.271755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:21.272023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:21.272751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] RBAC: Adding query filter params (role.id=97fb86a8759e4f148719d1d705a0d44c, scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0, user.id=e7bec1fe5f26425eb9d3e98f7502a7d0) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:21.273059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:21.274531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cc3a8-f29d-4a53-aca7-4b0417c46503 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:21.280364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 642/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Sep 13 19:48:21 2022] GET /identity/v3/role_assignments?user.id=e7bec1fe5f26425eb9d3e98f7502a7d0&scope.project.id=5597c130581c48e7bcd2fdd79c5e72e0&role.id=97fb86a8759e4f148719d1d705a0d44c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:23.511600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8742-23d3-49d9-9447-9853bf1308e0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:23.512220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8742-23d3-49d9-9447-9853bf1308e0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:23.512220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8742-23d3-49d9-9447-9853bf1308e0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:23.513499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 646/1288] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:23.519077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:23.519680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:23.519680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:23.539097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:23.540318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:23.572242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:23.612805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 643/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:23.881836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8742-23d3-49d9-9447-9853bf1308e0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:23.988671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8742-23d3-49d9-9447-9853bf1308e0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:24.000452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:24.026511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:24.027150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.027429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.027704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.028292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:24.029677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:24.040845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:24.042609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:24.043793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 647/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:48:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:24.048317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:24.151075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21198dc3-a85c-4c56-bde4-cc5d1824d846 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:24.189146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:24.189776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.190052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.190322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.190976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:24.191273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:24.192637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4ba3f-a705-40ea-bfdc-486792f53dbf None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:24.197578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 644/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Sep 13 19:48:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:24.205001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:24.318177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a4272-4e22-481e-a3c6-a8d0e3dcaacd None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:24.356135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:24.356747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.357029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.357296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] PATH_INFO: `/v3/services` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.357937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:24.358235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:24.359661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbbaf4-0de1-4c09-beef-56da6c875519 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:24.364275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 648/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Sep 13 19:48:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:26.510593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc4e0-e19b-4b52-9022-f3ef0e6ad2b4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:26.511187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc4e0-e19b-4b52-9022-f3ef0e6ad2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:26.511187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efc4e0-e19b-4b52-9022-f3ef0e6ad2b4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:26.512503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 645/1293] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:26.518206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d956-eb53-4ad7-9735-5c3e6557cc1a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:26.518729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d956-eb53-4ad7-9735-5c3e6557cc1a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:26.518866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d956-eb53-4ad7-9735-5c3e6557cc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:26.538691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2537d956-eb53-4ad7-9735-5c3e6557cc1a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:26.539885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2537d956-eb53-4ad7-9735-5c3e6557cc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:26.573382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:26.614191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 649/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:26.902643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efc4e0-e19b-4b52-9022-f3ef0e6ad2b4 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:26.940441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:27.013963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efc4e0-e19b-4b52-9022-f3ef0e6ad2b4 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:27.050032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:27.050674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:27.050955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:27.051229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:27.051884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:27.053262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f9e6e-8ce0-4365-abf6-183d767f26b0 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:27.071419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 646/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Sep 13 19:48:26 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:29.217854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89748-df05-4653-a0ab-2c9ba6afa54b None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.218437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89748-df05-4653-a0ab-2c9ba6afa54b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.218437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c89748-df05-4653-a0ab-2c9ba6afa54b None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.219825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 650/1296] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:29.225333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.225797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.225887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.246026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:29.247282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:29.322764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 647/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:29 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:29.587865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c89748-df05-4653-a0ab-2c9ba6afa54b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:29.694177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c89748-df05-4653-a0ab-2c9ba6afa54b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:29.736597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:29.737419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.737742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.738164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.738983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:29.740546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:29.750477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:29.751684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:29.752564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 651/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:48:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:48:29.757882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:29.777061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:29.863134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea6c62-6f53-466d-8ead-e2bb5b9386cf None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:29.898495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:29.899121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.899404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.899674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.900308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:29.900608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:29.901988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:29.907580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 648/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Sep 13 19:48:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:29.913859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:29.948339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:30.029817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac18b3-bab8-4d30-ba46-81df99de4a47 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:30.067755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:30.068375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:30.068652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:30.068915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:30.069652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] RBAC: Adding query filter params (interface=public, service_id=cf6ec911c6e84db796b64f8e3de3dd1d, region_id=RegionOne) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:30.069961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:30.071372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64914f90-bb30-4cbe-a3bf-abc3c0178d3d None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:30.077314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 652/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Sep 13 19:48:29 2022] GET /identity/v3/endpoints?service_id=cf6ec911c6e84db796b64f8e3de3dd1d&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:30.083469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:30.170683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:30.189891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a962ac-f62d-4508-a4e2-ee43f69f75d6 None admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:30.225346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:30.226047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:30.226220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:30.226509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:30.227472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:30.228838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0df072-95ac-4f95-8271-39a919acc441 None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:30.235905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 649/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Sep 13 19:48:30 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:32.366010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9fec-a7f5-4db8-b7fb-d8ee208d6528 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.366575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9fec-a7f5-4db8-b7fb-d8ee208d6528 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.366704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa9fec-a7f5-4db8-b7fb-d8ee208d6528 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.367984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 653/1302] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:32.373715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.374111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.374254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.393315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:32.394554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:32.466244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 650/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:48:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:48:32.741720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa9fec-a7f5-4db8-b7fb-d8ee208d6528 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:32.843746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:32.851566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa9fec-a7f5-4db8-b7fb-d8ee208d6528 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:32.892583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:32.893194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.893473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.893745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.894337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:32.895748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:48:32.905007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:48:32.906238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application with Lock( Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: ERROR keystone.server.flask.application Sep 13 19:48:32.907382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 654/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Sep 13 19:48:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:48:32.915019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:33.001443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:33.025400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6696a-a7d8-48e9-a172-3c5a6ef817a5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:33.059639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:33.060247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:33.060524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:33.060791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] PATH_INFO: `/v3/services` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:33.061417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:33.061720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:33.063147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1064735-2ee3-4cfc-b896-b05e043588fb None admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:33.068204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 651/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Sep 13 19:48:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:33.077952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:33.195434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0a19-60a4-45aa-8c9e-ba938486b682 None admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:33.230732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:33.231338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:33.231614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:33.231886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:33.232552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:33.233931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a722f9-c340-4354-816d-eb4fdc13b2a6 None admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:33.256956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 655/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:43.065281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=30 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:43.252883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:38.720675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a756-c870-4cc6-91bb-c16247d83f97 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.721961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a756-c870-4cc6-91bb-c16247d83f97 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.722433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a756-c870-4cc6-91bb-c16247d83f97 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.723983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 652/1307] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:49:38.729657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af408e2-8453-4aa7-b70d-4262626c737a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.730546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af408e2-8453-4aa7-b70d-4262626c737a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.730993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af408e2-8453-4aa7-b70d-4262626c737a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.851275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6af408e2-8453-4aa7-b70d-4262626c737a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:38.852534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6af408e2-8453-4aa7-b70d-4262626c737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.074752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 656/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:39.368331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.368544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.368824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.370247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 653/1309] 10.222.0.52 () {54 vars in 965 bytes} [Tue Sep 13 19:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:39.376960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cbc10-02c1-4244-92c1-113cbe2d6d66 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.377349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cbc10-02c1-4244-92c1-113cbe2d6d66 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.377699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cbc10-02c1-4244-92c1-113cbe2d6d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.461928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2e0cbc10-02c1-4244-92c1-113cbe2d6d66 None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:39.463555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2e0cbc10-02c1-4244-92c1-113cbe2d6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.490932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:39.598777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.600787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.610771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 657/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Sep 13 19:49:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:39.750749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12ade8-9867-4f0a-aa58-909e01bf7082 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.836256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.837129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.837547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.837953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.839826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.905770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:39.973574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.975082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd197c-af9c-42ab-abf7-e5c60c7c52ac service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.995769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 654/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:42.485780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.486442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.486442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.487717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 658/1312] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:42.493270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539799-33e3-49a9-ac44-90f52fc7f5ad None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.493722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539799-33e3-49a9-ac44-90f52fc7f5ad None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.493816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539799-33e3-49a9-ac44-90f52fc7f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.589583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-cf539799-33e3-49a9-ac44-90f52fc7f5ad None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:42.590853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cf539799-33e3-49a9-ac44-90f52fc7f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:42.692539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 655/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:42.968590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.970139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:43.093913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66265e-ff44-419b-b6a5-3f7f9569dbb3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.172206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:43.174419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.175257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.175542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.175823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.177987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:43.290577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.292033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2bd3c-fd0b-4bd3-b9ed-9cfc984f22a7 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.310612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 659/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:45.766022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d39130-0248-4602-b8c1-b58bedc01f53 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:45.766618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d39130-0248-4602-b8c1-b58bedc01f53 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:45.766618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d39130-0248-4602-b8c1-b58bedc01f53 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:45.767945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 656/1315] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:45.773881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbdad-ea48-4a74-8496-dc53baae8aaf None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:45.774361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbdad-ea48-4a74-8496-dc53baae8aaf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:45.774502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dbdad-ea48-4a74-8496-dc53baae8aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:45.841641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b86dbdad-ea48-4a74-8496-dc53baae8aaf None None] MFA Rules not processed for user `a418650541634431838ac8d10049c192`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:45.842810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b86dbdad-ea48-4a74-8496-dc53baae8aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:45.896653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:45.941975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 660/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:46.233180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.233450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.233886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.235665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 657/1317] 10.222.0.52 () {54 vars in 965 bytes} [Tue Sep 13 19:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:46.241952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44818d3-67a4-479e-9d3b-b5eb4ca61f1a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.242208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44818d3-67a4-479e-9d3b-b5eb4ca61f1a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.242537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44818d3-67a4-479e-9d3b-b5eb4ca61f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.322710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f44818d3-67a4-479e-9d3b-b5eb4ca61f1a None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:46.324031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f44818d3-67a4-479e-9d3b-b5eb4ca61f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:46.401654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 661/1318] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:46.410972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.412534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:46.422962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:46.522218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7935bd1-a758-42ff-93e9-40d14ef233b0 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.602384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.603040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.603313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.603577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.605189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:46.671139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:46.720531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.722353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d693d7b-3ecc-4416-a49c-7330ea403a6f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.741579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 658/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:49.801378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:49.801983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:49.801983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:49.803348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 662/1320] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:49.809692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1b0-e2b5-48ca-80a8-8eff1407e6dc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:49.810119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1b0-e2b5-48ca-80a8-8eff1407e6dc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:49.810233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1b0-e2b5-48ca-80a8-8eff1407e6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:49.880667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d868a1b0-e2b5-48ca-80a8-8eff1407e6dc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:49.881795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d868a1b0-e2b5-48ca-80a8-8eff1407e6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:49.981032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 659/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:50.259861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.261566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:50.313372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:50.369180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a92cb-3d28-499a-9b7b-65e7cff13bea None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.446077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.446721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.446996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.447253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.448870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:50.550990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:50.565978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.567517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6b052-27b2-412b-bdae-335e0201827c service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.588061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 663/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:53.538796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.539583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.539583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.541095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 660/1323] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:53.552916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b99b2-7ba8-4a0f-853b-c4c6a3245191 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.553703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b99b2-7ba8-4a0f-853b-c4c6a3245191 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.553840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b99b2-7ba8-4a0f-853b-c4c6a3245191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.645385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5a4b99b2-7ba8-4a0f-853b-c4c6a3245191 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:53.646671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5a4b99b2-7ba8-4a0f-853b-c4c6a3245191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:53.761258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 664/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:54.040165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:54.041869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:54.063215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:54.150595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bbf57-2ad6-4bad-afee-c9592d540e01 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:54.230658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:54.231294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:54.231559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:54.231837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:54.233454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:54.306533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:54.345865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:54.347362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcff06-b8d5-4469-9f44-3fe7a78bdb74 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:54.365509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 661/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:56.587644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.588235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.588477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.590013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 665/1326] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:56.596173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc495c04-44c4-4875-84e6-d34ea471f68b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.596715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc495c04-44c4-4875-84e6-d34ea471f68b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.596811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc495c04-44c4-4875-84e6-d34ea471f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.676687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bc495c04-44c4-4875-84e6-d34ea471f68b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:56.678018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bc495c04-44c4-4875-84e6-d34ea471f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.781981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 662/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:57.061118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.062704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.126256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:57.172151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbc214-4cf5-4aa4-8abf-e9e36077b578 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.255302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.256099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.256375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.256643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.258340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.384532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:57.387059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.388604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267b6bb-2d8e-4299-8f1f-0c579141d92a service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.408222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 666/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:59.618043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.618708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.618708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.620155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 663/1329] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:59.626480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5612e3-19f2-4a6e-a2c5-4dfdf3764181 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.627084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5612e3-19f2-4a6e-a2c5-4dfdf3764181 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.627084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5612e3-19f2-4a6e-a2c5-4dfdf3764181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.701402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8c5612e3-19f2-4a6e-a2c5-4dfdf3764181 None None] MFA Rules not processed for user `a418650541634431838ac8d10049c192`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:59.702579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8c5612e3-19f2-4a6e-a2c5-4dfdf3764181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:59.811932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 667/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:49:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:00.095612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:00.097254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.135018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:00.211313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd176f-55b6-4f54-8273-6fc3ca4f5049 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:00.291922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:00.292546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.292819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.293085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.294714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.383723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:00.447707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:00.450685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8079a9e-e9fc-49a5-9e30-c54e19817999 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:00.480466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 664/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:03.246338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.246985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.246985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.248533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 668/1332] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:03.254768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f953aec-2ece-4a52-9ecb-042a02182b99 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.255353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f953aec-2ece-4a52-9ecb-042a02182b99 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.255353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f953aec-2ece-4a52-9ecb-042a02182b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.329624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0f953aec-2ece-4a52-9ecb-042a02182b99 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:03.330908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0f953aec-2ece-4a52-9ecb-042a02182b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.439911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 665/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:03.731286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.732860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.819853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:03.861904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de0975-00c7-469d-98fe-fd0f93429f66 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.955286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.955954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.956235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.956543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.958252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.092341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.093816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02377c-8e87-4114-b657-565f4fba52c4 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.106359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:04.117675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 669/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:06.398196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.399029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.399029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.400210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 666/1335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:06.406970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e473e-6c02-449e-8744-41c635011258 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.407804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e473e-6c02-449e-8744-41c635011258 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.407997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e473e-6c02-449e-8744-41c635011258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.490355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-662e473e-6c02-449e-8744-41c635011258 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:06.492304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-662e473e-6c02-449e-8744-41c635011258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:06.619872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 670/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:06.917736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.919442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:06.967181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:07.046934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea3d0-50de-43ea-9eaa-6c91c198e4f9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.132620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:07.133249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.133526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.133793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.135577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:07.236732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:07.271589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.273454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4179e699-03e0-4aa7-8fe4-0f90735ed8cd service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.296621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 667/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:10.200883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.201546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.201546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.202885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 671/1338] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:10.209952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab576-ba55-48e6-a375-4e7a77ff0627 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.210562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab576-ba55-48e6-a375-4e7a77ff0627 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.210727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab576-ba55-48e6-a375-4e7a77ff0627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.290828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ee7ab576-ba55-48e6-a375-4e7a77ff0627 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:10.292091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ee7ab576-ba55-48e6-a375-4e7a77ff0627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:10.433999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 668/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:10.736927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:10.738856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:10.837160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:10.874707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735d4cb-6313-4f36-99ef-91f015fdbcb2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:10.962552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:10.963198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.963477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.963749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.965527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:11.099152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.100681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e519d1-2081-4eca-8df1-2070471238be service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.122680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 672/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:15.889738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.890627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.890627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.891722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 669/1341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:15.899001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc9ec5-84fd-4895-9789-a65dd43792c8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.899813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc9ec5-84fd-4895-9789-a65dd43792c8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.900251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc9ec5-84fd-4895-9789-a65dd43792c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.910328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.012801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d7cc9ec5-84fd-4895-9789-a65dd43792c8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.014066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d7cc9ec5-84fd-4895-9789-a65dd43792c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.138650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 673/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.428877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.430864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.501552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.593787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a82fb4-3f43-4a34-9fa7-84ccdec4f893 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.700074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.700838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.701232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.701724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.703615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.861665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.884504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.886212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27782294-cb7e-4ebe-b3e0-4513659e41ca service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.908555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 670/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:20.098265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.099093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.099093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.100278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 674/1344] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:20.108284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e855-02d3-4868-9626-4a51971c6715 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.109167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e855-02d3-4868-9626-4a51971c6715 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.109167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e855-02d3-4868-9626-4a51971c6715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.193057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9288e855-02d3-4868-9626-4a51971c6715 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:20.194546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9288e855-02d3-4868-9626-4a51971c6715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:20.380897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 671/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:20.895350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.898105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:21.036692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:21.074747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87e09e-23ff-44d6-9c02-ee977be09cf8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.183363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.184393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.184623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.185003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.186931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:21.388447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.390312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507409-11a1-4770-849a-58eb8089a1e0 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.418232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 675/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:26.708369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.709279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.709279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.710490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 672/1347] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:26.718650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3ab5-63db-49c9-bbc7-d65998b1d869 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.720746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3ab5-63db-49c9-bbc7-d65998b1d869 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.721179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e3ab5-63db-49c9-bbc7-d65998b1d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.743336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.825812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c43e3ab5-63db-49c9-bbc7-d65998b1d869 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:26.827050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c43e3ab5-63db-49c9-bbc7-d65998b1d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.978749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 676/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:27.345212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.346851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.428835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:27.497177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741605c-bc22-4b47-bb5c-dfbc1aeb8fa3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.596086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.596846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.597094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.597367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.599271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.796741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:27.801775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.803527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18bb02-fccf-4d1f-8cb6-3a54036c2488 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.828970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 673/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:34.638026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec9e0-7da1-4b31-98a9-c25d34ad304e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.638026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec9e0-7da1-4b31-98a9-c25d34ad304e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.638026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ec9e0-7da1-4b31-98a9-c25d34ad304e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.640223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 677/1350] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:50:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:34.652764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395901e6-f6ed-4c84-b3ab-58189f51a9f2 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.652764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395901e6-f6ed-4c84-b3ab-58189f51a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.652764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395901e6-f6ed-4c84-b3ab-58189f51a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.857243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-395901e6-f6ed-4c84-b3ab-58189f51a9f2 None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:34.860078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-395901e6-f6ed-4c84-b3ab-58189f51a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:35.195085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 674/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 536 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:36.976409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=59 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:41.682300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9818ef-adef-4e22-8324-89f077e226ca None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.683400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9818ef-adef-4e22-8324-89f077e226ca None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.683400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9818ef-adef-4e22-8324-89f077e226ca None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.686322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 678/1352] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:50:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:41.695228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580ab4a-f5b0-4f08-a5e9-8de61b296e7b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.696975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580ab4a-f5b0-4f08-a5e9-8de61b296e7b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.697783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580ab4a-f5b0-4f08-a5e9-8de61b296e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.848144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:41.897923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-6580ab4a-f5b0-4f08-a5e9-8de61b296e7b None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:41.900626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6580ab4a-f5b0-4f08-a5e9-8de61b296e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.128469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 675/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:43.036305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcdfef-4b17-4563-9610-4c5046eed308 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.037042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcdfef-4b17-4563-9610-4c5046eed308 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.037193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcdfef-4b17-4563-9610-4c5046eed308 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.039108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 679/1354] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:43.047477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2616d-fdca-4875-a9b4-7c91995ca62f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.048928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2616d-fdca-4875-a9b4-7c91995ca62f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.049725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2616d-fdca-4875-a9b4-7c91995ca62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.232597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-49d2616d-fdca-4875-a9b4-7c91995ca62f None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:43.234727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-49d2616d-fdca-4875-a9b4-7c91995ca62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:43.329513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:43.436221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 676/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:47.735883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fad8b-3196-4193-8462-1ba92b4d8e4d None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.736846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fad8b-3196-4193-8462-1ba92b4d8e4d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.738225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fad8b-3196-4193-8462-1ba92b4d8e4d None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.741083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 680/1356] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:47.747242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3148-849c-4363-adb2-69f271d7b142 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.747854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3148-849c-4363-adb2-69f271d7b142 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.748916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3148-849c-4363-adb2-69f271d7b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.912579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-230b3148-849c-4363-adb2-69f271d7b142 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.913826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-230b3148-849c-4363-adb2-69f271d7b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.084426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 677/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.394230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.394512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.394655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.396064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 681/1358] 10.222.0.52 () {54 vars in 962 bytes} [Tue Sep 13 19:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:48.406757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e5c0-b67e-4894-a8f5-aba49cc1b771 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.407614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e5c0-b67e-4894-a8f5-aba49cc1b771 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.408496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e5c0-b67e-4894-a8f5-aba49cc1b771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.440677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:48.535561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-64c8e5c0-b67e-4894-a8f5-aba49cc1b771 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.536799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-64c8e5c0-b67e-4894-a8f5-aba49cc1b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.771962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 678/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.782523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.785451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.014528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2ec7e-508d-40f1-a0cb-a53a94e14279 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.113091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:49.165046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.165706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.165961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.166232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.168013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.281775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.283517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e265dea-0db4-40fe-8628-23b3b17781bb service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.305894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 682/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:51.781786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.782564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.782564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.784174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 679/1361] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:51.790239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882da60c-1fb1-42fe-8b72-689f7c40b829 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.791113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882da60c-1fb1-42fe-8b72-689f7c40b829 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.791113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882da60c-1fb1-42fe-8b72-689f7c40b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.880301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-882da60c-1fb1-42fe-8b72-689f7c40b829 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:51.881647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-882da60c-1fb1-42fe-8b72-689f7c40b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:51.935437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.026885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 683/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:52.336062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.338869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.433963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.599572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3a125-06be-4bbd-9145-5c16e64201c4 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.791215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.792032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.792238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.792684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.794688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.824527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.920370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.922339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a8b85-7684-48ef-a347-11bfebf344fc service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.947541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 680/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:55.782145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd385b8a-bf0d-40d0-a30c-9bfb25a7e609 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.783275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd385b8a-bf0d-40d0-a30c-9bfb25a7e609 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.783275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd385b8a-bf0d-40d0-a30c-9bfb25a7e609 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.784167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 684/1364] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:55.790736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f644-bdc0-4c09-92db-fd07c1d5b620 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.791633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f644-bdc0-4c09-92db-fd07c1d5b620 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.791633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f644-bdc0-4c09-92db-fd07c1d5b620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.878444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e5e1f644-bdc0-4c09-92db-fd07c1d5b620 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.880032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e5e1f644-bdc0-4c09-92db-fd07c1d5b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.028585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:56.043120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 681/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:56.347573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.348212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.348798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.351952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 685/1366] 10.222.0.52 () {54 vars in 962 bytes} [Tue Sep 13 19:50:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:56.358261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1377-12c1-4be4-900d-83036cf873df None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.358600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1377-12c1-4be4-900d-83036cf873df None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.358894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b1377-12c1-4be4-900d-83036cf873df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.497205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b28b1377-12c1-4be4-900d-83036cf873df None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:56.498757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b28b1377-12c1-4be4-900d-83036cf873df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.665282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 682/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:56.674249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.677495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.952268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d33d6-0e6a-45cb-8968-f3a9f632416b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.981279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.160300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.161331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.161793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.162258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.165547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.361440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.364611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c7480-1977-4cc4-915e-84d870d47db8 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.401783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 686/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:57.761477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277171d-79cb-410e-a077-75da8897612e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.762085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277171d-79cb-410e-a077-75da8897612e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.762283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277171d-79cb-410e-a077-75da8897612e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.764399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 683/1369] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:57.771164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807cf4-a5cb-4b68-a695-f3bde57bc5aa None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.771599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807cf4-a5cb-4b68-a695-f3bde57bc5aa None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.772012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807cf4-a5cb-4b68-a695-f3bde57bc5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.814594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.867240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-59807cf4-a5cb-4b68-a695-f3bde57bc5aa None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:57.868997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-59807cf4-a5cb-4b68-a695-f3bde57bc5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.003924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 687/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:58.317288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.317997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.318651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.322398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 684/1371] 10.222.0.52 () {54 vars in 964 bytes} [Tue Sep 13 19:50:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:58.333631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdeb4-4bf9-4b95-8648-f0b02f82df7c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.334066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdeb4-4bf9-4b95-8648-f0b02f82df7c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.334527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbdeb4-4bf9-4b95-8648-f0b02f82df7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.452048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-18bbdeb4-4bf9-4b95-8648-f0b02f82df7c None None] MFA Rules not processed for user `f641d3de0a3a43ad8bc789483ffd5a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:58.453944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-18bbdeb4-4bf9-4b95-8648-f0b02f82df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.517739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.561594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 688/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:58.574662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.576565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.657354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.759755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445cfa9-a45a-409e-a0e1-8c8ad5ec3604 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.871528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.872563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.872563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.872563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.874246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.994850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.017570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.019772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdcb59-ca8d-4f1c-b246-340336d3cce5 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.046054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 685/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:00.316306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.317205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.318668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.321316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 689/1374] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:00.328957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc837-618f-4b89-b538-fd8d100a09f3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.329906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc837-618f-4b89-b538-fd8d100a09f3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.329906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdc837-618f-4b89-b538-fd8d100a09f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.416429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1cfdc837-618f-4b89-b538-fd8d100a09f3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.417927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1cfdc837-618f-4b89-b538-fd8d100a09f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.586591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 686/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.082418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.085318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.337513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e7584-ab4e-4d69-a9e3-cdb90ab839a2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.422159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.563235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.563791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.564338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.564803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.567777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.696584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.699149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ef52a-96f9-49ac-8db9-38282b542924 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.725642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 690/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:04.977850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.978776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.978776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.980777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 687/1377] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:04.989416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608d0d-c9db-4a7d-a614-465988393387 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.990259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608d0d-c9db-4a7d-a614-465988393387 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.990259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a608d0d-c9db-4a7d-a614-465988393387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.129075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0a608d0d-c9db-4a7d-a614-465988393387 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:05.131129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0a608d0d-c9db-4a7d-a614-465988393387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.145115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.318506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 691/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:05.609649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.611264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.684851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.794761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527b6fb-60a7-4d60-8cfc-6a6e9f0afb38 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.934525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.935153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.935424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.935688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.937404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.961263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.076554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.079623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aaeb35-e3f0-4416-9448-dedc239f17d0 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.116510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 688/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:08.705744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.706521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.707089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.709287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 692/1380] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:08.719430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2dca-4c61-4863-927e-b23087e13651 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.720838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2dca-4c61-4863-927e-b23087e13651 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.721668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2dca-4c61-4863-927e-b23087e13651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.880027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-483c2dca-4c61-4863-927e-b23087e13651 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:08.883882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-483c2dca-4c61-4863-927e-b23087e13651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.081113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.091482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 689/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:09.410837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.413738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.641238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.656105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d194a8-8eae-4dc3-8f2b-7c7bea83d2ff None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.810961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.811727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.811902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.812173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.814398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.946613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.950010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85f4f8-387d-41b0-a588-b5d4f2fe4d2b service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.969108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.984894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 693/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:12.907675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.908322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.908732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.911862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 690/1383] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:12.923023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ec37-aaff-44d0-ab36-c593fd45aa83 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.923725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ec37-aaff-44d0-ab36-c593fd45aa83 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.924465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8ec37-aaff-44d0-ab36-c593fd45aa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.064036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-dcc8ec37-aaff-44d0-ab36-c593fd45aa83 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.066299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dcc8ec37-aaff-44d0-ab36-c593fd45aa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.248230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 694/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:13.695522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.701819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.913392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175526a8-91b8-449a-a6d1-1ed3e4458a67 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.006310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:14.068379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.069066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.069607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.069607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.071647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.235831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.237502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6665e-0c60-4ba0-b27d-caf910016442 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.261711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 691/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:16.672571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f8762-aded-4017-a846-b57919053c0e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.673225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f8762-aded-4017-a846-b57919053c0e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.673225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f8762-aded-4017-a846-b57919053c0e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.675217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 695/1386] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:16.682159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.682897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.682897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.769338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.770687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.780315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fbd3-92cf-4bea-ad23-fa87f8d5c87e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.781058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fbd3-92cf-4bea-ad23-fa87f8d5c87e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.781339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660fbd3-92cf-4bea-ad23-fa87f8d5c87e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.783000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 696/1387] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:51:16.789010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b92c50-e472-42e2-9dac-269c02201811 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.789312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b92c50-e472-42e2-9dac-269c02201811 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.789538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b92c50-e472-42e2-9dac-269c02201811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.809538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:16.871344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c6b92c50-e472-42e2-9dac-269c02201811 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.872620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c6b92c50-e472-42e2-9dac-269c02201811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.890227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 692/1388] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.905616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.000998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 697/1389] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:17.248204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.249737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.316580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b812-091f-4ee2-ad2d-45a18d2e0fcf None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.317062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b812-091f-4ee2-ad2d-45a18d2e0fcf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.317483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b812-091f-4ee2-ad2d-45a18d2e0fcf None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.320101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 698/1390] 10.222.0.52 () {54 vars in 966 bytes} [Tue Sep 13 19:51:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:51:17.328650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.329226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.329637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.420523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94993f7a-93f0-4ef6-8984-bf0849e340f0 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.429685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] MFA Rules not processed for user `0d318e084073402e8d3385eb69c9685b`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:17.431164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.489831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.537568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 699/1391] 10.222.0.52 () {58 vars in 1063 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:17.551976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.562628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527eedf-e6c8-4a4c-9c72-d3b9287a3330 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.565548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.566184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.566541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.566770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.568675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.584855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.662794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.663434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.663715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.663987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.665711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.699523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.701189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6933c3-0d62-4980-9970-0e87ba1385d8 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.722386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 693/1392] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:17.823973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.825984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563bbb4-1c4f-4adb-a3e7-6a30b873e7be service glance] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.854670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 700/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:18.907237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05549be-ee0e-4f9c-85ac-b28fb5cff4d3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.907892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05549be-ee0e-4f9c-85ac-b28fb5cff4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.908217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05549be-ee0e-4f9c-85ac-b28fb5cff4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.031196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b05549be-ee0e-4f9c-85ac-b28fb5cff4d3 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.033312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b05549be-ee0e-4f9c-85ac-b28fb5cff4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.039207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.146546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 694/1394] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:19.210400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.211561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.211902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.213710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 701/1395] 10.222.0.52 () {54 vars in 943 bytes} [Tue Sep 13 19:51:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:19.222477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3669e5-407b-4fc9-b019-8439f0902d5f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.222784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3669e5-407b-4fc9-b019-8439f0902d5f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.223021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3669e5-407b-4fc9-b019-8439f0902d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.321419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8c3669e5-407b-4fc9-b019-8439f0902d5f None None] MFA Rules not processed for user `09cae6567bc540bab88dc59c1ddee27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.322729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8c3669e5-407b-4fc9-b019-8439f0902d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.462662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.521284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 695/1396] 10.222.0.52 () {58 vars in 1040 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:19.536946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.540087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.607419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.754977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade739-fec3-451b-a115-1a6dbd5927fc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.939032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.939809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.940181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.940550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.942960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.985767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:20.048681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa696-fb2c-4d55-b91e-55f43e9ec91f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:20.049518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa696-fb2c-4d55-b91e-55f43e9ec91f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:20.049518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa696-fb2c-4d55-b91e-55f43e9ec91f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:20.050895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 696/1397] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:51:20.056887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69985-62af-4e38-9c36-f31ccd2d8c6a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:20.057221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69985-62af-4e38-9c36-f31ccd2d8c6a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:20.057578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69985-62af-4e38-9c36-f31ccd2d8c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:20.061505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:20.063179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:20.072810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 702/1398] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:51:20.142756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-afd69985-62af-4e38-9c36-f31ccd2d8c6a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:20.144940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-afd69985-62af-4e38-9c36-f31ccd2d8c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:20.326631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 697/1399] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:20.658295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:20.661095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:20.863848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fcd23-59cb-4ca0-a16d-51f41a15dbfa service swift] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:21.018815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:21.108514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:21.109708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:21.110137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:21.110596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:21.112746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:21.285737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:21.288901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595cbdc-df10-4082-9b7a-588999e51cf5 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:21.321877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 703/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:24.073248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.074209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.074209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.075364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 698/1401] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:24.081701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a171-c0fc-4bea-a498-e3d2262005f3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.082292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a171-c0fc-4bea-a498-e3d2262005f3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.082292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a171-c0fc-4bea-a498-e3d2262005f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.180575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9b69a171-c0fc-4bea-a498-e3d2262005f3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:24.182536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9b69a171-c0fc-4bea-a498-e3d2262005f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.229700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:24.330555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 704/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:24.758247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.760228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.867485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:24.989891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7430692-1879-4372-8ee6-91abf3570139 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.142854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.144019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.144422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.145038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.148239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:25.189080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:25.321981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.325169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35f305-3b8c-453e-b1ca-7329a6193d78 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.373278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 699/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:28.009273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:28.009852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:28.009852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:28.014532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 705/1404] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:28.017457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf099-d626-4b3e-8f97-2a99596670de None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:28.018002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf099-d626-4b3e-8f97-2a99596670de None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:28.018002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febf099-d626-4b3e-8f97-2a99596670de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:28.129180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8febf099-d626-4b3e-8f97-2a99596670de None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:28.131794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8febf099-d626-4b3e-8f97-2a99596670de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:28.300405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:28.316371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 700/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:28.743005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:28.745886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:29.024779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86166c-3bd6-45da-a7d8-5d5866f6d516 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:29.064604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:29.211151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:29.212272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:29.212764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:29.213283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:29.215357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:29.354860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:29.356725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312602e-eefb-4604-906c-0570cec5df47 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:29.377141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 706/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:32.026450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.028406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.029603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.032801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 701/1407] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:51:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:32.043883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84ce8-155f-4b8e-9221-f248678338f5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.045189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84ce8-155f-4b8e-9221-f248678338f5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.045675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84ce8-155f-4b8e-9221-f248678338f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.106585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.169217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-94a84ce8-155f-4b8e-9221-f248678338f5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.171101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-94a84ce8-155f-4b8e-9221-f248678338f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.361478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 707/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.673235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.674871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.936717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9bd0-034d-49d4-87d8-6a719b1704d1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.091656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.262420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00516156-7854-4781-9863-a31261e57b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.283242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516156-7854-4781-9863-a31261e57b35 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.283242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516156-7854-4781-9863-a31261e57b35 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.283242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00516156-7854-4781-9863-a31261e57b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.283242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-00516156-7854-4781-9863-a31261e57b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.465780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00516156-7854-4781-9863-a31261e57b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.467487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00516156-7854-4781-9863-a31261e57b35 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.491970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 702/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:42.358137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=54 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:43.489617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=64 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:44.498868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24272a-70bd-432f-ac08-aab089a2ac7a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:44.499456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24272a-70bd-432f-ac08-aab089a2ac7a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:44.499456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24272a-70bd-432f-ac08-aab089a2ac7a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:44.500797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 708/1410] 10.222.0.52 () {54 vars in 918 bytes} [Tue Sep 13 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:44.507756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef30a04-bc39-4641-a52a-bdf27db6b642 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:44.508274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef30a04-bc39-4641-a52a-bdf27db6b642 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:44.508274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef30a04-bc39-4641-a52a-bdf27db6b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:44.600254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fef30a04-bc39-4641-a52a-bdf27db6b642 None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:44.601503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fef30a04-bc39-4641-a52a-bdf27db6b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:44.727973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 703/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Sep 13 19:51:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:44.743673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb73f5-beb4-4c87-ac41-12938dd5c242 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:44.744021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb73f5-beb4-4c87-ac41-12938dd5c242 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:44.744331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb73f5-beb4-4c87-ac41-12938dd5c242 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:44.745902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 709/1412] 10.222.0.52 () {54 vars in 918 bytes} [Tue Sep 13 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:44.752193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dad25-7537-469a-8ee1-319511508e82 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:44.752489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dad25-7537-469a-8ee1-319511508e82 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:44.752786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dad25-7537-469a-8ee1-319511508e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:44.837663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:44.841452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-351dad25-7537-469a-8ee1-319511508e82 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:44.843515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-351dad25-7537-469a-8ee1-319511508e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:44.977475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 704/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Sep 13 19:51:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:47.621615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:47.622550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:47.622550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:47.624321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:47.630942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313b1fd-a39b-4087-a020-5536f28c4501 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:47.631723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313b1fd-a39b-4087-a020-5536f28c4501 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:47.631723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313b1fd-a39b-4087-a020-5536f28c4501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:47.723454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5313b1fd-a39b-4087-a020-5536f28c4501 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:47.724676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5313b1fd-a39b-4087-a020-5536f28c4501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:47.796419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:47.850022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.150713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.152263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.316824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe4623-0561-4878-bfcc-9faaa7be30b7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.395073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:48.448107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.448726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.449001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.449268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.450918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.554078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.555598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cb538-30da-4b6e-b0fd-50dc43652400 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.574620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 711/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:49.768252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.768966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.768966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.770166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:49.778351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242a31-fd7a-46f7-852f-b9e8e0778105 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.779430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242a31-fd7a-46f7-852f-b9e8e0778105 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.779430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242a31-fd7a-46f7-852f-b9e8e0778105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.862633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e1242a31-fd7a-46f7-852f-b9e8e0778105 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:49.863769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e1242a31-fd7a-46f7-852f-b9e8e0778105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:49.891922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:49.969776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:50.132668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.134994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:50.300385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb9a1f-ebbc-45cd-b0fa-9610e316e968 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.342066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:50.418971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.419513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.419880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.420163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.421868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:50.526642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.528278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cde8e-f208-4254-bef4-80e40b4de1d6 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.547579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 707/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:51.704083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.704854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.704854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.706027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:51:51.713165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506ba2-a13d-4bcd-bac2-df522d2d4674 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.713800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506ba2-a13d-4bcd-bac2-df522d2d4674 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.713800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14506ba2-a13d-4bcd-bac2-df522d2d4674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.775843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:51.800870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-14506ba2-a13d-4bcd-bac2-df522d2d4674 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:51.802162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-14506ba2-a13d-4bcd-bac2-df522d2d4674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:51.921513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:52.074892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.076773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.256966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70b4fb-f280-41ba-8e5a-54cc038b5668 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.274432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:52.396781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.397423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.397740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.398003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.399756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.502122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.503880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cece6-4073-4a72-9653-886183eafa9e service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.524819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 714/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:53.661079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.661681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.661681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.663008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:53.670726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227b-acf3-4ec9-9fd3-f945b56c5b2c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.671305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227b-acf3-4ec9-9fd3-f945b56c5b2c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.671305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227b-acf3-4ec9-9fd3-f945b56c5b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.692958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:53.765384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4aac227b-acf3-4ec9-9fd3-f945b56c5b2c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:53.766630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4aac227b-acf3-4ec9-9fd3-f945b56c5b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.870200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:54.040683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:54.042213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:54.167097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:54.207089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a073-6bba-41b6-98c4-0c66a379a3ce None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:54.339947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:54.340775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:54.341032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:54.341389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:54.344013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:54.431748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:54.448524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:54.450190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bdf2e-4455-480d-bde9-2720f5b5938b service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:54.470718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 710/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:55.628478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:55.629062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:55.629062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:55.630419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:55.637556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608d5e-7118-4fe8-bba1-a7a4563560b1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:55.638115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608d5e-7118-4fe8-bba1-a7a4563560b1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:55.638115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608d5e-7118-4fe8-bba1-a7a4563560b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:55.721917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d2608d5e-7118-4fe8-bba1-a7a4563560b1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:55.723479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d2608d5e-7118-4fe8-bba1-a7a4563560b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:55.830694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:55.988334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:55.989953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:56.086445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:56.155714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb1386-0312-4cea-a4b4-cdfe0e980459 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:56.408973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:56.409505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:56.409756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:56.410081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:56.411729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:56.463348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:56.509304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:56.511012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a7948-33aa-4180-b65a-908cc24b9c62 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:56.531705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 717/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:57.686389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:57.686987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:57.686987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:57.688310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:57.695647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613add-2f13-474a-a16d-87a587eb05df None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:57.696455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613add-2f13-474a-a16d-87a587eb05df None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:57.696616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613add-2f13-474a-a16d-87a587eb05df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:57.778170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d4613add-2f13-474a-a16d-87a587eb05df None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:57.780028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d4613add-2f13-474a-a16d-87a587eb05df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:57.901975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:58.053016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:58.055620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:58.115289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:58.222685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180d914-3e21-42c1-ac15-6d0d84d43368 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:58.350180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:58.350838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:58.351118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:58.351386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:58.353023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:58.360515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:58.452774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:58.454564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe74679-3459-4278-ba98-d5b06b57bc0c service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:58.476517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 713/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:59.542098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.542710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.542710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.544075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:59.551233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2859e0-cbad-45ae-aa2a-9ffda4608910 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.551791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2859e0-cbad-45ae-aa2a-9ffda4608910 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.551791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2859e0-cbad-45ae-aa2a-9ffda4608910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.633383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1f2859e0-cbad-45ae-aa2a-9ffda4608910 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:59.634639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1f2859e0-cbad-45ae-aa2a-9ffda4608910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:59.710972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:59.743319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:59.896506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.898771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:59.927481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:00.081528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d4c1-c43c-4f34-b022-d6e345ee53ee None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:00.185449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:00.216159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:00.216818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:00.217093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:00.217359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:00.219082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:00.315085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:00.316767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e41d63-518d-4aa4-9e16-de8d8e3a0bee service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:00.335598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 720/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:01.443160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:01.444016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:01.444016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:01.446128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:52:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:01.455829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993815f-6f3e-409b-9f55-a04e8daaf87f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:01.456292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993815f-6f3e-409b-9f55-a04e8daaf87f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:01.456409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993815f-6f3e-409b-9f55-a04e8daaf87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:01.545394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-7993815f-6f3e-409b-9f55-a04e8daaf87f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:01.546696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7993815f-6f3e-409b-9f55-a04e8daaf87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:01.591866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:01.650719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:01.810608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:01.813170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:01.981773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642598f5-d824-40be-97aa-5817dd97bf5f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:02.044256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:02.109400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:02.110031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:02.110305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:02.110595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:02.112220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:02.335127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:02.336742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0d59c-35d3-414b-ba24-965d2c6fe984 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:02.356981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 716/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:03.432131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:03.432806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:03.432806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:03.434104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:03.441458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca449-5cde-4c59-b429-1104d85940bd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:03.442363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca449-5cde-4c59-b429-1104d85940bd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:03.442363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca449-5cde-4c59-b429-1104d85940bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:03.536053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-228ca449-5cde-4c59-b429-1104d85940bd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:03.537297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-228ca449-5cde-4c59-b429-1104d85940bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:03.539447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:03.643394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:03.794704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:03.796321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:03.967636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69b7ba-65f8-4441-9e42-1d664f7409a1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:03.997665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:04.091322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:04.091831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:04.092122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:04.092378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:04.093961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:04.203906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:04.206686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab72ceb6-964a-43d4-9a78-4dc4b8680e12 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:04.238919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 723/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:06.347791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.348716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.348716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.350755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:06.356433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd64d-de0f-45fe-8fc6-decb97e8fdc8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.356865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd64d-de0f-45fe-8fc6-decb97e8fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.356977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd64d-de0f-45fe-8fc6-decb97e8fdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.399824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:06.427140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-47fbd64d-de0f-45fe-8fc6-decb97e8fdc8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:06.428403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-47fbd64d-de0f-45fe-8fc6-decb97e8fdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:06.534285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:06.823131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:06.825714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:06.949390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6711ad7-e2a6-4123-8bf7-753123cb6516 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:06.988182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:07.035031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:07.035650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:07.035925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:07.036193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:07.037801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:07.162196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:07.163603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444034e5-22ee-491f-a925-acab61d3e9b6 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:07.182770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 719/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:09.540170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:09.540697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:09.540697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:09.542064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:09.547761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ade-e48d-4c49-a347-57d3205b89d2 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:09.548243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ade-e48d-4c49-a347-57d3205b89d2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:09.548243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ade-e48d-4c49-a347-57d3205b89d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:09.621510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-68272ade-e48d-4c49-a347-57d3205b89d2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:09.622787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-68272ade-e48d-4c49-a347-57d3205b89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:09.641374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:09.730181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:10.016366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.018623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.140746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbbc38-1005-42f3-8aed-0088b4a2ea2c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.155156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:10.225499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.226131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.226435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.226710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.228667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.352840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.354460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc71ddc-1b32-4f7a-bdb8-0978a278ced4 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.374541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 726/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:12.993732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:12.994290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:12.994290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:12.995671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:13.001957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1754-1931-4dbc-8f8d-da569aa9dc52 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.002476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1754-1931-4dbc-8f8d-da569aa9dc52 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.002585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d1754-1931-4dbc-8f8d-da569aa9dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.058708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:13.078866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-bd4d1754-1931-4dbc-8f8d-da569aa9dc52 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:13.080116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-bd4d1754-1931-4dbc-8f8d-da569aa9dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.200282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:13.488499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.490324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.607816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e55fd0-dc2b-4b98-a2c3-8e15267411e2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.650286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:13.687422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.688026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.688300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.688567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.690216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.842889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.846155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c057e-f8e1-48d3-ab3f-b8381d0d8069 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.867394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 722/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:16.293808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53249543-a445-4b79-8134-07d571a54721 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:16.294548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53249543-a445-4b79-8134-07d571a54721 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:16.294548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53249543-a445-4b79-8134-07d571a54721 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:16.295795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:16.302326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397be8-e166-4173-9934-49d90fee79b7 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:16.302969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397be8-e166-4173-9934-49d90fee79b7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:16.302969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397be8-e166-4173-9934-49d90fee79b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:16.382281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-83397be8-e166-4173-9934-49d90fee79b7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:16.383558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-83397be8-e166-4173-9934-49d90fee79b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:16.406625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:16.505082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:16.810659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53249543-a445-4b79-8134-07d571a54721 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:16.812293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-53249543-a445-4b79-8134-07d571a54721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:16.942402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53249543-a445-4b79-8134-07d571a54721 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:16.964908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:17.034002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:17.034662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:17.034941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:17.035212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:17.036923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:17.155130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:17.156594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f477b80-ad0d-4fb5-9fdf-c0dd28678812 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:17.174858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 729/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:19.430934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:19.431794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:19.431794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:19.433856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:19.443212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c443d-ba1b-4abb-9942-3a082065dfa2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:19.443973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c443d-ba1b-4abb-9942-3a082065dfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:19.444151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c443d-ba1b-4abb-9942-3a082065dfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:19.518489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:19.522676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e06c443d-ba1b-4abb-9942-3a082065dfa2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:19.524947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e06c443d-ba1b-4abb-9942-3a082065dfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:19.627931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:19.913546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:19.915671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:20.041517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67524f2-746b-40cc-8556-e83ee0e3af9e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:20.095616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:20.135076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:20.135810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:20.136316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:20.136567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:20.138965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:20.272913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:20.274441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c5195-9560-4a18-b6d7-9c486ed8c45e service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:20.296021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 725/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:22.569234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:22.569794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:22.569794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:22.571201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:22.576871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19789-f9db-477b-8833-a7377cfcbfec None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:22.577312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19789-f9db-477b-8833-a7377cfcbfec None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:22.577427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19789-f9db-477b-8833-a7377cfcbfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:22.650407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-15a19789-f9db-477b-8833-a7377cfcbfec None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:22.651668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-15a19789-f9db-477b-8833-a7377cfcbfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:22.683916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:22.758122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:23.039905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:23.041554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:23.155630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a920079-5973-40bd-a62a-97c498fdc1e0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:23.187791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:23.237974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:23.238623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.238898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.239181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.240787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:23.372792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:23.374155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4992924a-1312-4c57-a160-7e9b48e592fa service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:23.393874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 732/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:25.708354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a633c-475d-4f32-911c-11913a437088 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:25.708964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a633c-475d-4f32-911c-11913a437088 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:25.708964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a633c-475d-4f32-911c-11913a437088 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:25.710265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:52:25.717238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3394-174a-4169-a798-3606021bb03b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:25.717811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3394-174a-4169-a798-3606021bb03b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:25.718184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf3394-174a-4169-a798-3606021bb03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:25.806616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cdcf3394-174a-4169-a798-3606021bb03b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:25.807976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cdcf3394-174a-4169-a798-3606021bb03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:25.817600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:25.925963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:26.214249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a633c-475d-4f32-911c-11913a437088 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.215878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-341a633c-475d-4f32-911c-11913a437088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.351791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a633c-475d-4f32-911c-11913a437088 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.413564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:26.436515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.437151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.437429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.437701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.439565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.592910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.596058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80355dfd-38d6-4b90-95c8-3f268abf2a1e service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.632560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 728/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:28.963607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.964243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.964537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.967134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:52:28.976267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a88a-bb91-48e5-b315-ffaca23887a6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.976963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a88a-bb91-48e5-b315-ffaca23887a6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.977159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583a88a-bb91-48e5-b315-ffaca23887a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.060164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d583a88a-bb91-48e5-b315-ffaca23887a6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:29.061412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d583a88a-bb91-48e5-b315-ffaca23887a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.104142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:29.173265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:29.462322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:29.464005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.606899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832ecb2-81a1-43b8-8213-17f04a5f99b3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:29.651887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:29.701690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:29.702324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.702632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.702898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.704622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.828148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:29.829664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c792f-4049-4a8c-a230-d184982de851 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.851834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 735/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:32.104067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.104634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.104634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.106045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:32.112337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f577d-011b-4de4-8547-9bc162903dc1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.113239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f577d-011b-4de4-8547-9bc162903dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.113239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f577d-011b-4de4-8547-9bc162903dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.188405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-608f577d-011b-4de4-8547-9bc162903dc1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:32.189718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-608f577d-011b-4de4-8547-9bc162903dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.209808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:32.297295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:32.579493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:32.581081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.708923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc8899-8b5d-4457-9222-a85428983a3d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:32.800722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:32.809201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:32.809834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.810129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.810401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.812102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:33.013235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:33.015816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af511b7-760f-4e0f-ae79-39d1821a0ed1 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:33.054086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 731/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:35.373846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:35.374516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:35.374516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:35.375764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:35.381649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f36f-bc74-4d0a-86d6-dfacc53d385c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:35.382142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f36f-bc74-4d0a-86d6-dfacc53d385c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:35.382257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7f36f-bc74-4d0a-86d6-dfacc53d385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:35.478055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-90a7f36f-bc74-4d0a-86d6-dfacc53d385c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:35.479318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-90a7f36f-bc74-4d0a-86d6-dfacc53d385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:35.532801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:35.605179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:35.905069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:35.906648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:36.034695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e0083-135b-4a40-bb0f-528ca87cc9bf None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:36.089865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:36.120831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:36.121462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:36.121744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:36.122018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:36.123689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:36.247797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:36.249874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff620-87ea-43bc-bc37-e2d2da99b689 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:36.270536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 738/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:45.600003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=22 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:46.268315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=66 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:57.310243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990655f-54d1-4244-9650-b92ba4c6873d None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:57.311157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990655f-54d1-4244-9650-b92ba4c6873d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:57.311157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990655f-54d1-4244-9650-b92ba4c6873d None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:57.312811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 733/1471] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:57.321393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a28b-6a6f-4ed4-a13b-78ed93f3cdbf None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:57.322049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a28b-6a6f-4ed4-a13b-78ed93f3cdbf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:57.322049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a28b-6a6f-4ed4-a13b-78ed93f3cdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:57.398332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4096a28b-6a6f-4ed4-a13b-78ed93f3cdbf None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:57.399611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4096a28b-6a6f-4ed4-a13b-78ed93f3cdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:57.506014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 739/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:57.814327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:57.814652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:57.814924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:57.816350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 734/1473] 10.222.0.52 () {54 vars in 966 bytes} [Tue Sep 13 19:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:52:57.822795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca159cc-7721-4006-bd6a-a562045a717e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:57.823041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca159cc-7721-4006-bd6a-a562045a717e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:57.823401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca159cc-7721-4006-bd6a-a562045a717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:57.916737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cca159cc-7721-4006-bd6a-a562045a717e None None] MFA Rules not processed for user `0d318e084073402e8d3385eb69c9685b`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:57.917962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cca159cc-7721-4006-bd6a-a562045a717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:57.941248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:58.006565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 740/1474] 10.222.0.52 () {58 vars in 1063 bytes} [Tue Sep 13 19:52:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:58.014964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:58.019282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:58.139902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bce3bb-b063-4290-a04d-6660ec3dd904 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:58.229382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:58.230023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:58.230301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:58.230598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:58.232211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:58.289159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:58.348353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:58.350181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993201b-a2ce-4a77-9709-3c7d65953b57 service glance] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:58.370814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 735/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:00.704728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.705488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.705488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.706753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 741/1476] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:00.715578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eca1dd-1c1b-4455-bc29-5858ade4368b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.716322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eca1dd-1c1b-4455-bc29-5858ade4368b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.716514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eca1dd-1c1b-4455-bc29-5858ade4368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.796860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-58eca1dd-1c1b-4455-bc29-5858ade4368b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:00.798022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-58eca1dd-1c1b-4455-bc29-5858ade4368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:00.910018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 736/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:01.188025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.189877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:01.337861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc5737-ce37-4c12-91c2-8fb92e0ed74c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.396330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:01.448990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.449521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.449797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.450068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.450728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.452525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:53:01.452525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:53:01.452525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:53:01.453661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e656d5-72e1-4c2c-a8f8-30a03198097f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.474386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 742/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Sep 13 19:53:01 2022] GET /identity/v3/projects => generated 2553 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:03.712886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.713481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.713481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.714899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 737/1479] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:03.724879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cea3d-19b8-41c1-a985-713669a104f7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.725506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cea3d-19b8-41c1-a985-713669a104f7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.725837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cea3d-19b8-41c1-a985-713669a104f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.818748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c65cea3d-19b8-41c1-a985-713669a104f7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:03.820011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c65cea3d-19b8-41c1-a985-713669a104f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.940960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 743/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:04.263682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.265286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:04.276023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:04.445246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14f39-dce4-47ef-87bb-4156b0ed8f0a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.534072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:04.574208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.574413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.574685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.574978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.576661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:04.682097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.683886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3cfc8-ecd0-40b8-82e5-60c64a4da512 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.703536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 738/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:07.223573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:07.224537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:07.224537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:07.225842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 744/1482] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:07.231849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400af15-4a57-402e-8e0b-39adeb94e6ed None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:07.232494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400af15-4a57-402e-8e0b-39adeb94e6ed None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:07.232494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400af15-4a57-402e-8e0b-39adeb94e6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:07.316314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4400af15-4a57-402e-8e0b-39adeb94e6ed None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:07.317631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4400af15-4a57-402e-8e0b-39adeb94e6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:07.370289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:07.428715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 739/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:07.723090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:07.725169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:07.755861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:07.907364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbad50-267b-4d1e-983f-547334a65364 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:08.045767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:08.046822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:08.047147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:08.047461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:08.049544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:08.149536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:08.198293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:08.199814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303ec7-e9fe-4266-9548-b2c857a9bed4 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:08.221405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 745/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:10.605688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:10.606497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:10.606723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:10.609408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 740/1485] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:10.617923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd95bb-3669-4d88-9b68-73c519d640f4 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:10.618533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd95bb-3669-4d88-9b68-73c519d640f4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:10.618533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd95bb-3669-4d88-9b68-73c519d640f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:10.709071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-06bd95bb-3669-4d88-9b68-73c519d640f4 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:10.710344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-06bd95bb-3669-4d88-9b68-73c519d640f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:10.858642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 746/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:11.147747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:11.150043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:11.297508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a680-deb3-4506-8748-8e217bb6348b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:11.391049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:11.392737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:11.393441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:11.393731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:11.394253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:11.396902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:11.539777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:11.541864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3d127-4ede-4b56-a6ef-4ac11bf578c4 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:11.564741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 741/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:14.020279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:14.020856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:14.020856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:14.022197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 747/1488] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:14.028067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562914-4b80-4ac1-97fd-d22879e8ab7d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:14.028580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562914-4b80-4ac1-97fd-d22879e8ab7d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:14.028580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd562914-4b80-4ac1-97fd-d22879e8ab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:14.107186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bd562914-4b80-4ac1-97fd-d22879e8ab7d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:14.108426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bd562914-4b80-4ac1-97fd-d22879e8ab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:14.163956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:14.210129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 742/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:14.497921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:14.499785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:14.539590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:14.619578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42037071-2018-476c-9c7b-fdbacdf0d394 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:14.701361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:14.701987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:14.702268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:14.702575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:14.704287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:14.796878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:14.840191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:14.841717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a667ef-a213-40e2-908b-cc98993c0715 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:14.863462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 748/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:17.303989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:17.304561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:17.304561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:17.305883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 743/1491] 10.222.0.52 () {54 vars in 927 bytes} [Tue Sep 13 19:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:17.312392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0df0c5-d67f-460c-a2d7-0c871c9257c2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:17.312968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0df0c5-d67f-460c-a2d7-0c871c9257c2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:17.312968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0df0c5-d67f-460c-a2d7-0c871c9257c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:17.387268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3a0df0c5-d67f-460c-a2d7-0c871c9257c2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:17.388511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3a0df0c5-d67f-460c-a2d7-0c871c9257c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:17.492470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 749/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Sep 13 19:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:17.785101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:17.787380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:17.904382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92cfc6-dbcd-4d26-959f-01cc5c970f60 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:17.990472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:17.991092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:17.991367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:17.991635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:17.993294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:18.003651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:18.139654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:18.141145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4ed54-d03b-401e-8510-d244f567b5f3 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:18.161256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 744/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:27.490800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=82 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:28.158716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=50 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:34.455614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:34.456245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:34.456245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:34.543031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:34.544381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:34.669642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 750/1494] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:34.678329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:34.678871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:34.678986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:34.765617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:34.766925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:34.876664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 745/1495] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:34.882326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:34.884175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:34.901799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8c74f-c6a9-4ff5-924e-aa7ec68f6fdc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:34.973841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:35.008251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.008855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.009146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.009416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.010520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:54:35.010830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.011865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.011865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:54:35.011865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:54:35.012766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.024309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 751/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:54:35.036651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:35.046287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c6882-a6cc-4107-bf5b-dc9910b43d2d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:35.112161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:35.154010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.154707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.154982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.155242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.155875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.157635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.157635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:54:35.157635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:54:35.158900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.220048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 746/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:54:35 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:54:35.224900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:35.226497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:35.262274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198cc8d-54a2-4696-9ab4-a4cd97c34d43 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:35.360708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.361326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.361606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.361878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.362546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.363572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.363572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:54:35.363572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:54:35.364880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.408083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 752/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:54:35 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:54:35.413340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:35.415968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:35.486661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b608f-d70d-4553-8241-6f2f2006cf9a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:35.591614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:35.605618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.606239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.606525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.606841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.607484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.608492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.608492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:54:35.608492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:54:35.609426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.623928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 747/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:54:35.628139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:35.629734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:35.641469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf2b00-a4d6-4496-80cb-2e7d224018cf admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:35.670273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:35.761399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.762030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.762307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.762601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.763254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.764285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.764285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:54:35.764285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:54:35.765209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c27738-4a6b-429e-85cc-765802ec6d40 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.782764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 753/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:54:35.787825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:35.789417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:35.828416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8840530-eeff-4d41-9ad8-6fda6aad498c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:35.928342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:35.929022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:35.929299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:35.929564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] PATH_INFO: `/v3/projects/78836f9634e641a8a8651367b861275e/users/70c1ae47d0c242439da835cee7f886a4/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:35.954035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78836f9634e641a8a8651367b861275e, user_id=70c1ae47d0c242439da835cee7f886a4, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:35.955966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:35.955966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:54:35.955966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:54:35.958458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ed306-721c-4fa9-b6a6-f54e5b4bfa4b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:35.987652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 748/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:54:35 2022] PUT /identity/v3/projects/78836f9634e641a8a8651367b861275e/users/70c1ae47d0c242439da835cee7f886a4/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:36.000483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a948a3-d3c8-469b-852a-fcad7a12eb9c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.000770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a948a3-d3c8-469b-852a-fcad7a12eb9c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.001070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a948a3-d3c8-469b-852a-fcad7a12eb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.063253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-21a948a3-d3c8-469b-852a-fcad7a12eb9c None None] MFA Rules not processed for user `70c1ae47d0c242439da835cee7f886a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:36.064485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-21a948a3-d3c8-469b-852a-fcad7a12eb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.144586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 754/1502] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:54:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:36.158432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.158745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.159041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.160390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 749/1503] 10.222.0.52 () {54 vars in 964 bytes} [Tue Sep 13 19:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:54:36.165580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d586b-cce7-4041-9caf-90e265f935e3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.165868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d586b-cce7-4041-9caf-90e265f935e3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.166138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d586b-cce7-4041-9caf-90e265f935e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.200155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:36.249266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-742d586b-cce7-4041-9caf-90e265f935e3 None None] MFA Rules not processed for user `f641d3de0a3a43ad8bc789483ffd5a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:36.250630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-742d586b-cce7-4041-9caf-90e265f935e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.342070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 755/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Sep 13 19:54:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:54:36.356148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:36.358171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.417366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:36.485203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815644d-ae75-43d1-8e9c-23f64639c44f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:36.568221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:36.568452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.568567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.568850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.570654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.678883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:36.680284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:36.683798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:36.700240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 750/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:36.759717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7cfa7-4c5e-48cd-aec0-baddad92a4dc None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.760312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7cfa7-4c5e-48cd-aec0-baddad92a4dc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.760643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7cfa7-4c5e-48cd-aec0-baddad92a4dc None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.763059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 756/1506] 10.222.0.52 () {52 vars in 830 bytes} [Tue Sep 13 19:54:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:54:36.767372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:36.768623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.887798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec6b9-131b-4631-bcb7-2cc381585fb9 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:36.998003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:36.998666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.998937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.999202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] PATH_INFO: `/v3/users/70c1ae47d0c242439da835cee7f886a4` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:37.012333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c1ae47d0c242439da835cee7f886a4)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:37.013361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:37.013361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:54:37.013361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:54:37.014600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:37.030455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:37.094845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c1ae47d0c242439da835cee7f886a4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:37.109084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c1ae47d0c242439da835cee7f886a4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:37.137262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c1ae47d0c242439da835cee7f886a4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:37.137766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c1ae47d0c242439da835cee7f886a4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:37.154814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c1ae47d0c242439da835cee7f886a4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:37.161116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 751/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:54:36 2022] DELETE /identity/v3/users/70c1ae47d0c242439da835cee7f886a4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:37.165787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:37.166072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:37.166346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:37.227739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:37.228974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:37.308444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:37.325123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 757/1508] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:37.335143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:37.336434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:37.453139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa30d-a244-43a9-b142-7104001677b2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:37.514351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:37.542472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:37.543115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:37.543392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:37.543665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:37.545362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:37.675644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:37.677229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fed61b-ca5e-471f-9e96-1c47aa234e1a service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:37.698387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 752/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:38.071185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:38.072456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:38.084239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0769b2d-99e6-4efc-827c-279e08c02cc1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:38.203449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:38.204375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:38.204733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:38.205101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] PATH_INFO: `/v3/projects/78836f9634e641a8a8651367b861275e` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:38.217053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78836f9634e641a8a8651367b861275e)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:38.218098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:54:38.218098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:54:38.218098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:54:38.219341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:38.284372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78836f9634e641a8a8651367b861275e', 'request_id': 'req-374c0082-303e-4615-9e75-b41381cf950b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:38.290429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78836f9634e641a8a8651367b861275e', 'request_id': 'req-374c0082-303e-4615-9e75-b41381cf950b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:38.303768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78836f9634e641a8a8651367b861275e', 'request_id': 'req-374c0082-303e-4615-9e75-b41381cf950b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:38.330316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] The token cache is being invalidate because project 78836f9634e641a8a8651367b861275e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:54:38.330567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:38.330993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-374c0082-303e-4615-9e75-b41381cf950b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:38.332467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 758/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:54:38 2022] DELETE /identity/v3/projects/78836f9634e641a8a8651367b861275e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:47.695702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=62 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:48.328215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:10.938344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.939127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.939127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.940299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:10.947461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3ec9-5392-403c-95c5-e0db315fb7d2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.948107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3ec9-5392-403c-95c5-e0db315fb7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.948249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3ec9-5392-403c-95c5-e0db315fb7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.050026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-38aa3ec9-5392-403c-95c5-e0db315fb7d2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:11.051310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-38aa3ec9-5392-403c-95c5-e0db315fb7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:11.167462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:11.324219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.326581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:11.466294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857a91a-09d0-4b83-9a6d-f8814db6b1fc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.476617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.477669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.478177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.478688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.494855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.497959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6532-bbe7-4162-8254-c037c2afbbd9 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.515569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 754/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:56:12.677152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870701-16a5-47c3-bdd4-4a85506bea3e None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:12.677775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870701-16a5-47c3-bdd4-4a85506bea3e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:12.677775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870701-16a5-47c3-bdd4-4a85506bea3e None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:12.679237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:56:12.687676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffe1c-4a95-4a88-9960-a17383303d9c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:12.688278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffe1c-4a95-4a88-9960-a17383303d9c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:12.688438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffe1c-4a95-4a88-9960-a17383303d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:12.709640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-677ffe1c-4a95-4a88-9960-a17383303d9c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:12.710908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-677ffe1c-4a95-4a88-9960-a17383303d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:12.719674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:12.876545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870701-16a5-47c3-bdd4-4a85506bea3e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:12.893378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870701-16a5-47c3-bdd4-4a85506bea3e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:12.899099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:12.899720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:12.899994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:12.900259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:12.909218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:12.910860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c67e3b-1403-41a3-ab41-bf1d24a05486 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:12.920853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 761/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:14.260991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21ee-ac70-4e33-9dbb-4002c054449a None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:14.261600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21ee-ac70-4e33-9dbb-4002c054449a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:14.261818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21ee-ac70-4e33-9dbb-4002c054449a None None] PATH_INFO: `/v3` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:14.263673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 756/1517] 10.222.0.49 () {54 vars in 925 bytes} [Tue Sep 13 19:56:14 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:14.270672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ed2e0-341d-46a8-b09d-fd1fc3a01f62 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:14.271269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ed2e0-341d-46a8-b09d-fd1fc3a01f62 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:14.271269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ed2e0-341d-46a8-b09d-fd1fc3a01f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:14.344870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f21ed2e0-341d-46a8-b09d-fd1fc3a01f62 None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:14.346087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f21ed2e0-341d-46a8-b09d-fd1fc3a01f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:14.368936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:14.436223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 762/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Sep 13 19:56:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:18.115776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651b4f2-d489-4b83-81a0-f05e81fb9c85 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:18.116332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651b4f2-d489-4b83-81a0-f05e81fb9c85 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:18.116332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651b4f2-d489-4b83-81a0-f05e81fb9c85 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:18.117675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 757/1519] 10.222.0.52 () {54 vars in 961 bytes} [Tue Sep 13 19:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:18.122972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60888e9-f93d-41c4-8753-c10b5f1630a0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:18.123522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60888e9-f93d-41c4-8753-c10b5f1630a0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:18.123522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60888e9-f93d-41c4-8753-c10b5f1630a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:18.149057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f60888e9-f93d-41c4-8753-c10b5f1630a0 None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:18.150178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f60888e9-f93d-41c4-8753-c10b5f1630a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:18.159762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 763/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Sep 13 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:18.167763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651b4f2-d489-4b83-81a0-f05e81fb9c85 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:18.189496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651b4f2-d489-4b83-81a0-f05e81fb9c85 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:18.196624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:18.197262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:18.197535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:18.197801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:18.206867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:18.208483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b49a3-caa3-4845-9a96-2ff03394d16c service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:18.223372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 758/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:24.472204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd812-960f-491e-b3a5-403e2875d925 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.473466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd812-960f-491e-b3a5-403e2875d925 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.473466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd812-960f-491e-b3a5-403e2875d925 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.475095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:24.483228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69def8f-97b0-4d6d-884f-b3bf2a27e9f3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.483794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69def8f-97b0-4d6d-884f-b3bf2a27e9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.484073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69def8f-97b0-4d6d-884f-b3bf2a27e9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.512912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d69def8f-97b0-4d6d-884f-b3bf2a27e9f3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:24.514128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d69def8f-97b0-4d6d-884f-b3bf2a27e9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:24.522800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:24.686524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd812-960f-491e-b3a5-403e2875d925 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.702858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd812-960f-491e-b3a5-403e2875d925 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:24.708988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:24.709634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.709914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.710184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.719835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:24.721584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41f5ee-0709-434e-855a-fbaf4ef21dcf service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:24.733091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 765/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:25.884583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f98a-2356-4b35-88ea-254f170b9f0e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.885202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f98a-2356-4b35-88ea-254f170b9f0e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.885202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6f98a-2356-4b35-88ea-254f170b9f0e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.886681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:25.894064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f8cc-48fe-424d-ade0-0572b791f921 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.894752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f8cc-48fe-424d-ade0-0572b791f921 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.894752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f8cc-48fe-424d-ade0-0572b791f921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.918473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4270f8cc-48fe-424d-ade0-0572b791f921 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:25.919704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4270f8cc-48fe-424d-ade0-0572b791f921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:25.928195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:26.088836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6f98a-2356-4b35-88ea-254f170b9f0e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:26.113406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6f98a-2356-4b35-88ea-254f170b9f0e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.122358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.123378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.123786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.124287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.139441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.142252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f69237-1e0f-48c6-b8e7-266d8910a8bc service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.160032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 761/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:30.295609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:30.296506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:30.296506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:30.298220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:30.311584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83000936-22e3-4fac-91f9-e6da9016540b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:30.312123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83000936-22e3-4fac-91f9-e6da9016540b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:30.312123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83000936-22e3-4fac-91f9-e6da9016540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:30.335047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-83000936-22e3-4fac-91f9-e6da9016540b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:30.336286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-83000936-22e3-4fac-91f9-e6da9016540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:30.344896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:30.667157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:30.670523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:30.762130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3d023-6392-43b0-9cdd-c774b709c2b9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:30.766949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:30.768000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:30.768331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:30.768692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:30.779475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:30.781815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b32e2-1176-4705-9517-dd97ad0ddaeb service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:30.796744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 768/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:33.038788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:33.039533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:33.039533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:33.041692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:33.051182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caee21-7f38-4c0e-8853-031a8647fd1e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:33.051961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caee21-7f38-4c0e-8853-031a8647fd1e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:33.051961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caee21-7f38-4c0e-8853-031a8647fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:33.074928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c4caee21-7f38-4c0e-8853-031a8647fd1e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:33.076309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c4caee21-7f38-4c0e-8853-031a8647fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:33.084829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:33.409176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:33.411300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:33.431295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff7a38-8809-4036-a9ec-f03f3a2dae58 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:33.436788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:33.437766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:33.438248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:33.438789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:33.449743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:33.451210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd2a41-5eb6-4df8-a73e-dc9c8f9bb2d4 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:33.461534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 764/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:37.208641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:37.209724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:37.209724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:37.211157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:37.219936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57217d0-09c7-4d0e-accf-b21023849879 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:37.220512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57217d0-09c7-4d0e-accf-b21023849879 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:37.220953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57217d0-09c7-4d0e-accf-b21023849879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:37.248439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d57217d0-09c7-4d0e-accf-b21023849879 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:37.249610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d57217d0-09c7-4d0e-accf-b21023849879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:37.258382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:37.562882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:37.564649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:37.638583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:37.652153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e44f7-3d1a-4fc6-8fd4-452bb3b87dea None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:37.657108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:37.657500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:37.657777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:37.658063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:37.667955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:37.669421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd41262-df0e-4249-8eec-90b6f1e886cc service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:37.679084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 771/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:39.839419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:39.839984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:39.839984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:39.841247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:39.848461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43c1d-2b1f-4d18-8a2b-a7f7d14a48a4 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:39.849081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43c1d-2b1f-4d18-8a2b-a7f7d14a48a4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:39.849081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43c1d-2b1f-4d18-8a2b-a7f7d14a48a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:39.869952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c5d43c1d-2b1f-4d18-8a2b-a7f7d14a48a4 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:39.871235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c5d43c1d-2b1f-4d18-8a2b-a7f7d14a48a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:39.879914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:56:40.176723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:40.178499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:40.186328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=84 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:40.193709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42680c5c-e90a-4c74-b8d6-7b4c303bad93 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:40.197822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:40.198459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.198768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.199018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.210089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:40.211634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434919e-e920-45c7-a26a-4ce8ade93cca service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:40.223553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 767/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:49.864288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=11 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:50.218202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:07.377624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3065e-9f26-49e8-ae02-86e61ddbdb1e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.377624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3065e-9f26-49e8-ae02-86e61ddbdb1e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.377624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3065e-9f26-49e8-ae02-86e61ddbdb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.382531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b273a-5998-4178-8cc9-d2c4b4c3d6e6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.383488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b273a-5998-4178-8cc9-d2c4b4c3d6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.383488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b273a-5998-4178-8cc9-d2c4b4c3d6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.408918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-abc3065e-9f26-49e8-ae02-86e61ddbdb1e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.414209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-abc3065e-9f26-49e8-ae02-86e61ddbdb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.414618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e41b273a-5998-4178-8cc9-d2c4b4c3d6e6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.415836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e41b273a-5998-4178-8cc9-d2c4b4c3d6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.430304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 773/1540] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.432340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 768/1541] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:07.440240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.440905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.441414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.441802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.442097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.442437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.475428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.477096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.477708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.484057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.499509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 774/1542] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.500518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 769/1543] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:07.505011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.505254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.521487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0d9a4-62f6-4cb1-b970-830ab012b25a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.524649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29acb8-b8f9-4d45-8771-4d20a210440b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.549324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.550235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.550235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.551313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.551313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:07.551656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.552691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:07.552691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:58:07.552691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:58:07.553648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.556164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.557860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.560241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.560822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.561735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:07.562495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.565306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.589279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 770/1544] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:58:07.601100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.605225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 775/1545] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:07.614749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.617416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce019e-eb23-4a39-8dbf-a1d2adc8bbaf admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.631117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e461769-7282-48cd-83e5-5f529efc2ae6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.642450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.643144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.643428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.643711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.644420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.646494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358fb5a-954d-4373-8e73-5a771d3ebc52 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.649942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.650636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.650921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.651195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.651859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.652910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:07.652910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:58:07.652910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:58:07.654125 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518c87a-96e6-42cf-90ac-be550001a2a4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.730174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 771/1546] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:07.731842 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69f0-f83e-4073-9718-e78edd42b823 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.732135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69f0-f83e-4073-9718-e78edd42b823 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.732440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69f0-f83e-4073-9718-e78edd42b823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.735677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 776/1547] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:07.737511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcab90-ba8f-4013-b2b0-1e198025593c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.737798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcab90-ba8f-4013-b2b0-1e198025593c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.738073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcab90-ba8f-4013-b2b0-1e198025593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.755436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fe2c69f0-f83e-4073-9718-e78edd42b823 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.756740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fe2c69f0-f83e-4073-9718-e78edd42b823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.763067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a9dcab90-ba8f-4013-b2b0-1e198025593c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.764442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a9dcab90-ba8f-4013-b2b0-1e198025593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.837867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 772/1548] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.839349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.839620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.839876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.854474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 777/1549] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.854474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.854474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.854474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.866896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.868923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.874563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.875784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.883291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 773/1550] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.884578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.885183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 778/1551] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.886003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.898033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168d09-aa64-4bd3-bcb4-d76ff1c7e4b7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.900263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26d745-83b8-4215-96d3-13329ce121e2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.902931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.903566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.903847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.904122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.904918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.905176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.905804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.905956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:07.905956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:58:07.905956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:58:07.906218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.906347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.907032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.907223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc06f7-f2a7-43b2-b614-981f89e53248 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.909047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0108737-3b16-4513-a169-f0e70dc4ef21 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.932991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 774/1552] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:07.934270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98bcdb-e11e-4d93-9f68-5853b47d492b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.934582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98bcdb-e11e-4d93-9f68-5853b47d492b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.934857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98bcdb-e11e-4d93-9f68-5853b47d492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.937941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 779/1553] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:07.938589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe118-0c8c-4261-ab7a-4040dfd76bb1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.938878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe118-0c8c-4261-ab7a-4040dfd76bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.939150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe118-0c8c-4261-ab7a-4040dfd76bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.955746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4e98bcdb-e11e-4d93-9f68-5853b47d492b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.957013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4e98bcdb-e11e-4d93-9f68-5853b47d492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.961316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9eefe118-0c8c-4261-ab7a-4040dfd76bb1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.962736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9eefe118-0c8c-4261-ab7a-4040dfd76bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.965915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 775/1554] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.967354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a800-9f05-4fe0-b264-0844a9cc1040 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.967641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a800-9f05-4fe0-b264-0844a9cc1040 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.967913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a800-9f05-4fe0-b264-0844a9cc1040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.975647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 780/1555] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:07.977156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56feb8-dca6-4915-8f18-ac88e48f2bf3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.977435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56feb8-dca6-4915-8f18-ac88e48f2bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.977708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56feb8-dca6-4915-8f18-ac88e48f2bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.991942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5cb1a800-9f05-4fe0-b264-0844a9cc1040 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:07.993181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5cb1a800-9f05-4fe0-b264-0844a9cc1040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.006796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 776/1556] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.006796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.006796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.006796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.008466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ae56feb8-dca6-4915-8f18-ac88e48f2bf3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.009733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ae56feb8-dca6-4915-8f18-ac88e48f2bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.018320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 781/1557] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.019619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.019906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.020290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.027343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.028719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.038240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 777/1558] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.044607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.044809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.044809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.051855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 782/1559] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.054116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.055925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d182c-77a1-4ecc-8509-543ac37eef23 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.060306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.060936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.061212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.061511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.062189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.063845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d6790-d809-47e7-ab6a-4097922a4eab admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.072243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec9080-db72-499a-ac74-d57cb3fe7e83 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.074686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 778/1560] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:08.076262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.076545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.076820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.080669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.081733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.082244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.082795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.084000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.087180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b796117-eadc-4a4e-a93a-af2a3a7ec2d9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.097596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.098882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.100045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 783/1561] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:08.101500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.101788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.102061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.112701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 779/1562] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.114275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.123720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.124995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.127112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360c39-5955-4468-9ef9-77402b0ac4bd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.132207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.132827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.133102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.133448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.133615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 784/1563] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.134028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.134326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.134550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.136011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.144222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b197783-5190-485b-9265-4487bedac0c0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.149265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.149918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.150180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.150322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 780/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.150558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.151154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.151344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.151647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.153157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.169419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd93b7-7f82-4c06-9c45-fe4948f2d3f4 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.171249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 785/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.172500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.182804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.183460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.183877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.184150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.184843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.185148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.186852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.189484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c5dcc-4121-4465-859c-4cbf33235f97 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.195343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.195679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.195945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.196223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.196893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.197200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.198891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.202253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 781/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.203835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.212967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 786/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.214180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.218860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b043582-64e5-482c-a9c8-8ed4febeca4d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.225440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae670b3-273a-493d-b742-84c9987bed63 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.226106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.227273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.227744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.228234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.229245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.230787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.231305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.231440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.231726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.231948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.232607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.234456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.244798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 787/1568] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:08.245751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.250599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 782/1569] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:08.251557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.253181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.258018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.259792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea84129-0453-4c0d-81a3-bbbbae6a4ce3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.262835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804ca1-5527-4f1f-8add-7c400e82d169 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.266791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.267445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.267750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.268072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.268847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.269149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.269838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.270504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.270785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.270940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.271105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.271707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.272010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.273663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.284861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 788/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.285801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.295430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 783/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.295430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.301263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc8246-a46a-4fa8-9a3f-130d4bbed743 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.306178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.306476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e046a-0ceb-4dd5-9484-f513be2cc7b9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.306862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.307144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.307420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.308077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.310062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.311759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.312398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.312733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.312939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.313586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.315627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.344990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 789/1572] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.346771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.354002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 784/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 389 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.355434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.441759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd11e8-fcde-4ede-b0ed-4d60ac8c7f24 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.446359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e3a4a6-460c-4e40-9499-5fda2f386c63 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.447820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.448658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.448951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.449217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.449868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.451404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.451872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.452037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.452316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.452583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.453258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.455240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.487487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 790/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 405 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.488431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.491375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 785/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.492227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.563549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510fd8-c5ee-4572-b022-783999c92192 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.564025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542ddfd-e9ea-4ae4-813f-f68c799424ee admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.568690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.568922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.569402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.569579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.569696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.569868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.569988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] PATH_INFO: `/v3/projects/3fdc25924a1446f883d7993a9d4cc5b9/users/848f3bab2f8b404aaf6eca4beeb78487/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.570135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] PATH_INFO: `/v3/projects/4a5ad97df0834dc796dcfab5005f1918/users/3a2fc997f07745538606b91a3d1d9162/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.587293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fdc25924a1446f883d7993a9d4cc5b9, user_id=848f3bab2f8b404aaf6eca4beeb78487, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.589186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5ad97df0834dc796dcfab5005f1918, user_id=3a2fc997f07745538606b91a3d1d9162, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.590006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.590971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:08.590971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:58:08.590971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:58:08.593523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.612737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 791/1577] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:08 2022] PUT /identity/v3/projects/4a5ad97df0834dc796dcfab5005f1918/users/3a2fc997f07745538606b91a3d1d9162/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:08.614257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.618508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 786/1577] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:08 2022] PUT /identity/v3/projects/3fdc25924a1446f883d7993a9d4cc5b9/users/848f3bab2f8b404aaf6eca4beeb78487/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:08.619403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.669674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.680790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.696893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be02c6-c4d9-4bde-b244-449510de9985 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.707872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846509f-b4bc-4d1c-b29a-3cf1f3c1822a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.713630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.714478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.714763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.715197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.716291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.720438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.752790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 792/1578] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.753658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.775775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.776550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.777023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.777503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.778631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.781583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.816409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 787/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.816977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.840020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417331e-5e72-47eb-afb7-73b60adab311 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.899311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635b5e0-3c74-4aa3-bb87-4319d160cb45 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.905572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.906545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.907018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.907454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.908543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.908659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.909624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.910053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.910536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.911588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.912080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.915044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.942414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 788/1580] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.943694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.948712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 793/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.948712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.957664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e64ce1-d511-472c-ae4f-11cbb2109a6a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.957954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b39d22-4958-4041-a4c5-3827ea46e3f0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.963017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.963287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.963658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.963947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.964188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.964295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.964561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.964860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.964995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.965647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.966877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfa2eb-4432-42e2-adba-a6b0f420152a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.968020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c5abb-a0a4-447f-bd03-2e398d80c638 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.994661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 794/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/users => generated 303 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.996235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.996525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.996808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 789/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.996943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.999057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.999559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.000022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.033763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] MFA Rules not processed for user `3a2fc997f07745538606b91a3d1d9162`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:09.035236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.040827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] MFA Rules not processed for user `848f3bab2f8b404aaf6eca4beeb78487`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:09.042959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.073919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.092885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.118082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 790/1584] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:09.119090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.133141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d71a9-84cf-4b1d-a7a3-f0e27bf7a212 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.136713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 795/1585] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:09.137737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.138542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.139186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.139484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.139721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.140601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.142591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.151568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486b846-ba92-4be6-886d-ae998e1e403f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.157235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.157740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.158018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.158420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.159028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.161032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.177357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 791/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/users => generated 297 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.179349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.193239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 796/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/users => generated 289 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.193239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.194724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21f7ac-c0c7-412b-878d-3c567dae0f75 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.200898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.201161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7475c-3ded-49c8-b9ac-be17ac0bd0d9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.201614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.201921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.202189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.202977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.205934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.206108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.206591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.206872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.207145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.207802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.210158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.222352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 797/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.223515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.229570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 792/1589] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.229570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.235829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1c84-3786-4dab-a1e0-a088bdf02bb3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.241537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.243065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.243348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.243624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.244286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.245906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefe53-7ae1-4889-b7e8-020cb965b6d7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.246104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.254924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.255566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.255846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.256117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.256773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.258749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670459f-cd76-4c57-82fa-f2701b83b56b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.263800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 798/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.264677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.277682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 793/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.277806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573853-b863-4f8d-9d7d-8a5c48d65868 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.280840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e36a1-1d2e-4c45-ab2f-c0895c9effa5 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.281123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e36a1-1d2e-4c45-ab2f-c0895c9effa5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.281419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e36a1-1d2e-4c45-ab2f-c0895c9effa5 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.282395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.282848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 794/1592] 10.222.0.52 () {54 vars in 962 bytes} [Tue Sep 13 19:58:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:09.283050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.283350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.283639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.283743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e36a1-1d2e-4c45-ab2f-c0895c9effa5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.284285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.286236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.299476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e36a1-1d2e-4c45-ab2f-c0895c9effa5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.305397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.306164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.306616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.306998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.307955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.310237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.317418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 799/1593] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.318259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.324367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 795/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.325239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.336296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550aafd-7420-49b6-83ac-6f6e5a21de4b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.393160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5705cc7-87c1-4da9-9736-03336845a6af admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.398139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.398754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.399029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.399317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.400163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.403179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.411971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.412470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 800/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.414061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.414061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.414061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.414601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.414709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.415823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.426361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddb464-ad24-483b-b024-ab8fe77a3e94 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.426670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 796/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.427577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.433627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.434275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.434591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.434865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.435519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.437188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.438316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839aa455-166a-4fea-88b6-73f39762e9a1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.445957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.446998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.447646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.447927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.448231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 801/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.449091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.449228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.452107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84698853-97b2-41fa-b1e6-f50b5d63514f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.459288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f1c90-49a1-41ee-b5a1-5be6dfc64a48 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.464664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.465295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.465570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.466013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.466695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.468307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.468864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 797/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.470807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.471147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.471415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.482865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 802/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.483728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.494161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47a3e1-7c73-4796-8993-827d385d9d6f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.499038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.499663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.499940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.500215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.500883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.502930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.513349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:09.514632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.536185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 803/1600] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/users => generated 301 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.536185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.550456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189d13d-a137-4b52-9c82-4262d7bb603e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.555706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.556431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.556774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.557110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.557869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.559879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.575380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 804/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.576321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.585149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.587298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc5fc5-730f-428a-b09e-8492cb14b7b7 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.593105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.593751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.594418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.594687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.595350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.597070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.609298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 798/1602] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:09.610414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.611192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 805/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.614350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.621578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.626610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0393e8d-5902-4963-957a-20dbad11f6e2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.634598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.635001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a25bb6-5857-4391-9556-d0a1ecc6bd87 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.635299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.635579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.635847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] PATH_INFO: `/v3/projects/fdc8b1b561f247ed84babfad941e55f6/users/b07b8b251aaa456c9916c41f9a0b77e3/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.642483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.643687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.644184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.644668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] PATH_INFO: `/v3/projects/36a9fc0618bd410483d03d26b4ce7e27/users/3bedf6fd67ce4df9845597cdde102ee4/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.651434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc8b1b561f247ed84babfad941e55f6, user_id=b07b8b251aaa456c9916c41f9a0b77e3, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.654138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.663051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a9fc0618bd410483d03d26b4ce7e27, user_id=3bedf6fd67ce4df9845597cdde102ee4, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.666133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.669016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 806/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/fdc8b1b561f247ed84babfad941e55f6/users/b07b8b251aaa456c9916c41f9a0b77e3/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.669832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.684551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 799/1605] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/36a9fc0618bd410483d03d26b4ce7e27/users/3bedf6fd67ce4df9845597cdde102ee4/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.685530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.741788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc950-42eb-4014-9f11-f2119e365954 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.756972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfa1cb-b969-479d-b2ac-7e3e6df666c0 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.762889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.763661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.763891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.764122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] PATH_INFO: `/v3/projects/d7bedd256d43440b9733d405dd98ad03/users/71e3deb691c24bc6927d09f38aa26d64/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.782329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bedd256d43440b9733d405dd98ad03, user_id=71e3deb691c24bc6927d09f38aa26d64, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.785186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.800302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 800/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/d7bedd256d43440b9733d405dd98ad03/users/71e3deb691c24bc6927d09f38aa26d64/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.800718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.810808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.811802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.812085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.812396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] PATH_INFO: `/v3/projects/f483c5fd42bd4655bc9a2bbeb49efb1a/users/f54f79e394174315a1f0df895d888364/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.829016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f483c5fd42bd4655bc9a2bbeb49efb1a, user_id=f54f79e394174315a1f0df895d888364, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.832038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.847514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 807/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/f483c5fd42bd4655bc9a2bbeb49efb1a/users/f54f79e394174315a1f0df895d888364/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.848367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.877901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd026f05-c798-46a4-991e-3d168f4c1dfa admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.914806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57fb87-1435-4bf4-99c3-f9777dc8db1d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.919631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.920276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.920560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.920863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] PATH_INFO: `/v3/projects/23873d995d564c0388a4649e57821c69/users/2577fc11946c4597857187ae7b464fe7/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.927618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.937694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=23873d995d564c0388a4649e57821c69, user_id=2577fc11946c4597857187ae7b464fe7, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.939852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.940591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.940711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.940991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.941343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.942007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.942980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.943937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.959784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 801/1608] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.959784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.960197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 808/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/23873d995d564c0388a4649e57821c69/users/2577fc11946c4597857187ae7b464fe7/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.960197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.971736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fa4ae-7eb2-4463-a806-a17a9ca24c5b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.038197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.038785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.039225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.039346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] PATH_INFO: `/v3/projects/8ac3cf4af51e419c848daba7adc86024/users/bd55568bf88346b794922ef8555f9654/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.050347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f2b6-f790-4b6e-865d-6f897c122f5b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.055937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac3cf4af51e419c848daba7adc86024, user_id=bd55568bf88346b794922ef8555f9654, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.056318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.056960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.057231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.057503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.058913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c02c85-6d2c-46ab-8d16-c08abddb35fc admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.075421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 802/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/8ac3cf4af51e419c848daba7adc86024/users/bd55568bf88346b794922ef8555f9654/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:10.076629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534381c-6234-4923-874c-5dd91ce584a5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.077424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534381c-6234-4923-874c-5dd91ce584a5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.077776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534381c-6234-4923-874c-5dd91ce584a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.097831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d534381c-6234-4923-874c-5dd91ce584a5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.099121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d534381c-6234-4923-874c-5dd91ce584a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.118891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.120824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b7c62-5783-405f-a187-2319de939d49 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.132180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 809/1611] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:10.133624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b68d4-c7ae-46a7-b5b5-0dbab9463844 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.133915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b68d4-c7ae-46a7-b5b5-0dbab9463844 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.134185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b68d4-c7ae-46a7-b5b5-0dbab9463844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.165844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9b0b68d4-c7ae-46a7-b5b5-0dbab9463844 None None] MFA Rules not processed for user `3bedf6fd67ce4df9845597cdde102ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.167306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9b0b68d4-c7ae-46a7-b5b5-0dbab9463844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.176559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 803/1612] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.176559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.176559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.176559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.209301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] MFA Rules not processed for user `71e3deb691c24bc6927d09f38aa26d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.210578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.258646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 810/1613] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.258646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3586a4-93fe-4978-942e-5fd70cabfa7d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.258646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3586a4-93fe-4978-942e-5fd70cabfa7d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.258646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3586a4-93fe-4978-942e-5fd70cabfa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.289489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:10.295147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ac3586a4-93fe-4978-942e-5fd70cabfa7d None None] MFA Rules not processed for user `f54f79e394174315a1f0df895d888364`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.296466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ac3586a4-93fe-4978-942e-5fd70cabfa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.300435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 804/1614] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.300435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.306296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:10.311902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865b994-ac4f-47ea-9bfa-f4223d939c83 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.316919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.317746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.318167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.318717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] PATH_INFO: `/v3/projects/1742a0542ad242b8812f7acb59e27c8b/users/68ab3bd47e494b2697be2d29ae3c369a/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.334565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1742a0542ad242b8812f7acb59e27c8b, user_id=68ab3bd47e494b2697be2d29ae3c369a, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.337462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f3fb7-25e6-47a3-b807-54d3e3f9c67b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.353128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 805/1615] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:10 2022] PUT /identity/v3/projects/1742a0542ad242b8812f7acb59e27c8b/users/68ab3bd47e494b2697be2d29ae3c369a/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:10.357691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.358072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.358514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.370010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 811/1616] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.371306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.371644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.371909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.383852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.385391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.392828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.394070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.450818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 806/1617] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.451668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.456584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 812/1618] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:10.457552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.472608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc3bc4-1606-4303-8ef6-20394de9998a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.477563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.478215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.478490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.478795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.479437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.481405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.510813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 813/1619] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:10.511491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.517563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c9ab-5a66-49cd-8df5-f3f8ac239bc9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.569024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.569677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.570040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.570316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.577531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463bea0-556d-4cd3-97c2-3977a4892ad1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.579643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.581146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.583582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.584209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.584545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.584820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.585461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.587540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1e1d-ac31-4d7d-a384-91f724faf348 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.591359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 807/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:10.592362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.603368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d02023-c9bd-4bea-91c9-7e6a8ec7bc2b service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.609206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.609971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.610287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.610701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.611436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.613281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.618818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 814/1621] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:10.620108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9e9d-3d68-41e1-adc0-111744be90a4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.620245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9e9d-3d68-41e1-adc0-111744be90a4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.620511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e9e9d-3d68-41e1-adc0-111744be90a4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.622085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 815/1622] 10.222.0.52 () {54 vars in 912 bytes} [Tue Sep 13 19:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:10.623231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c924-9eb9-4b1c-8cfb-e3d373f033fb None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.623477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c924-9eb9-4b1c-8cfb-e3d373f033fb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.623726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144c924-9eb9-4b1c-8cfb-e3d373f033fb None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.624973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 816/1623] 10.222.0.52 () {54 vars in 962 bytes} [Tue Sep 13 19:58:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:10.625685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144c924-9eb9-4b1c-8cfb-e3d373f033fb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.632335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 808/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:10.633240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.643736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:10.644507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bc51c-06b5-437f-a2dc-226883523b8a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.670491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:10.696480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144c924-9eb9-4b1c-8cfb-e3d373f033fb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.700931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.701541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.701920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.702161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.715998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.716680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.716954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.717219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.776223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.777800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.791633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 817/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:10.792564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.804005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae357b5-21f3-47d7-87a5-b3b5682122d3 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.809809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.810458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.810921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.811305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.812025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.814263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.819563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.821423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1879ae7-8df0-4ded-a6c7-6b751990fea0 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.841781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 818/1626] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:10.843260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:10.852635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 809/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:10.854069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.854357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.854649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.856473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed65db-2449-45e1-a51b-fa1755c72e34 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:10.862278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:10.863006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.863281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.863554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.864245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:10.866190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b855c7-1b3c-438f-981d-16f330c61910 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:10.878531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.880445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.897721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 819/1628] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/users => generated 308 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:10.897721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:10.897721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:10.897721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:10.922338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:10.924733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:10.998893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 810/1629] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:11.007633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.020456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3a1d8-856c-4e7c-9276-9b67e1041977 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.024750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 820/1630] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Sep 13 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:11.025984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.026997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.027325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.028027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] PATH_INFO: `/v3/projects/1742a0542ad242b8812f7acb59e27c8b/users/68ab3bd47e494b2697be2d29ae3c369a/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.030311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1742a0542ad242b8812f7acb59e27c8b, user_id=68ab3bd47e494b2697be2d29ae3c369a, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.033943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.035791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.048085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 811/1631] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:10 2022] PUT /identity/v3/projects/1742a0542ad242b8812f7acb59e27c8b/users/68ab3bd47e494b2697be2d29ae3c369a/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:11.048676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.058042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e883a-ebd4-4753-9af5-28abd23da9e6 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.081359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:11.147638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:11.156442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.156838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.157289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.157612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.158339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.160376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.161215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e1a13-485e-440e-97c9-1208c140f8c8 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.168275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.169003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.169354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.169680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.170442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.172201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.177297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 821/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:11.177297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.184101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 812/1633] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:11.184922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.252368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7fb7a-cbba-4f8f-b3cc-08432fefc999 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.269580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.270494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.271031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.271391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.276263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7f1a3-3ba8-4122-a685-3ee4caf934d9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.285142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.285844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.286127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.286461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.287438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.290102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f47df-9a14-4cb8-8e79-1efd07243cd7 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.323907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 822/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:11.328868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.330232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.331542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.380089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] MFA Rules not processed for user `68ab3bd47e494b2697be2d29ae3c369a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:11.382013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:11.407080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.409087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.421225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 813/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:11.422197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.435171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda0146-a805-4429-8a1c-75ae5688e21b service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.440988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.442247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.442657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.443363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] PATH_INFO: `/v3/projects/c1cf0736b8a54e31ba478c01d4991716/users/dfb5982aa7d14e0088c329dc5a65c628/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.465187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cf0736b8a54e31ba478c01d4991716, user_id=dfb5982aa7d14e0088c329dc5a65c628, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.468467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.490050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 814/1636] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:11 2022] PUT /identity/v3/projects/c1cf0736b8a54e31ba478c01d4991716/users/dfb5982aa7d14e0088c329dc5a65c628/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:11.491699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.575497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 823/1637] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:11.576948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.622676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b64674-41f9-4e5a-93c0-aa67b4306495 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.680881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e711f-003b-4ea0-99bc-627f76115188 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.693179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.694847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.698130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.698850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] PATH_INFO: `/v3/projects/5bedf7fd94364396b1984d248a753aa2/users/b22bfce9baf64a24bcfa033f55095a39/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.702617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:11.711710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:11.746198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bedf7fd94364396b1984d248a753aa2, user_id=b22bfce9baf64a24bcfa033f55095a39, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.747156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:11.748248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:11.749269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:11.749269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:11.752847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:11.754330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.755956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:11.781173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 824/1638] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:11.783392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.794961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 815/1639] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:11 2022] PUT /identity/v3/projects/5bedf7fd94364396b1984d248a753aa2/users/b22bfce9baf64a24bcfa033f55095a39/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:11.796415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:11.922053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c8c18-1731-452f-8dd1-76ec9db01bc3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:11.989261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e38946-e717-44be-b57c-783c4bfb10c3 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:12.015547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.018912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.020631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.022747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] PATH_INFO: `/v3/projects/c1cf0736b8a54e31ba478c01d4991716/users/dfb5982aa7d14e0088c329dc5a65c628/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.030004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cf0736b8a54e31ba478c01d4991716, user_id=dfb5982aa7d14e0088c329dc5a65c628, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.037053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.068961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.069941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.070546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 816/1640] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:11 2022] PUT /identity/v3/projects/c1cf0736b8a54e31ba478c01d4991716/users/dfb5982aa7d14e0088c329dc5a65c628/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:12.070696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.071151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.071725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:12.183257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf1b6c-f9ce-407d-abf9-a478b9353867 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:12.190270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.191255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.191633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.191911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.192833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.195718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5877-df39-4087-9b11-beae3a0dc56e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.214289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 817/1641] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:12.216875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937e38a-3e5d-47ec-b9a9-98f831445a0e None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.217161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937e38a-3e5d-47ec-b9a9-98f831445a0e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.218408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.222406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.226241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937e38a-3e5d-47ec-b9a9-98f831445a0e None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.229897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 818/1642] 10.222.0.52 () {54 vars in 912 bytes} [Tue Sep 13 19:58:12 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:12.233012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ebd5-ccc3-4f21-b361-b3a54a4a021b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.233203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ebd5-ccc3-4f21-b361-b3a54a4a021b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.233505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ebd5-ccc3-4f21-b361-b3a54a4a021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.260668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 825/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:12.261413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:12.286235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b6d65-1455-4293-a816-90844b88514b service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:12.301244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4f47ebd5-ccc3-4f21-b361-b3a54a4a021b None None] MFA Rules not processed for user `dfb5982aa7d14e0088c329dc5a65c628`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:12.302678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4f47ebd5-ccc3-4f21-b361-b3a54a4a021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:12.315456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.316317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.316554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.317014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] PATH_INFO: `/v3/projects/5bedf7fd94364396b1984d248a753aa2/users/b22bfce9baf64a24bcfa033f55095a39/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.319954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bedf7fd94364396b1984d248a753aa2, user_id=b22bfce9baf64a24bcfa033f55095a39, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.333494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885e954-a685-4420-bb4f-34134ba35f27 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.389338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 826/1644] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:12 2022] PUT /identity/v3/projects/5bedf7fd94364396b1984d248a753aa2/users/b22bfce9baf64a24bcfa033f55095a39/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:12.394854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.395148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.395429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.416468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:12.435192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 819/1645] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:12.439781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.441356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.441851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.483721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:12.484994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:12.511914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] MFA Rules not processed for user `b22bfce9baf64a24bcfa033f55095a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:12.513879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:12.584929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:12.685774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 827/1646] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:58:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:12.687071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:12.747183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 820/1647] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:12 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:12.754089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:12.822075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadc102-5e3f-4cde-9d3a-02652a90fe58 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:12.830462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.831973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.832289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.832561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.883530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163858e2-b9b1-4f50-ba25-c92b56f8685c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:12.895411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:12.897333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:12.897656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:12.897911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] PATH_INFO: `/v3/users/3a2fc997f07745538606b91a3d1d9162` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:12.901169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2fc997f07745538606b91a3d1d9162)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.908956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.952942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:12.956942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:12.992795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 828/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:12.993964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:13.001500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2fc997f07745538606b91a3d1d9162'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:13.009118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df0357-2d38-4076-9f58-218533eaa36b service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:13.018622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2fc997f07745538606b91a3d1d9162'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:13.018890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:13.019008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:13.019367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:13.019645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:13.051426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:13.053319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:13.073236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2fc997f07745538606b91a3d1d9162'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:13.073640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 829/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:13.075359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2fc997f07745538606b91a3d1d9162'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:13.089620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b6c7804a-10ee-4aec-b9de-67ae26d83498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2fc997f07745538606b91a3d1d9162'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:13.094952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:13.100315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 821/1650] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:12 2022] DELETE /identity/v3/users/3a2fc997f07745538606b91a3d1d9162 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:13.106341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:13.106824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab28a7-4fdc-4287-b489-f96a3761ff68 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:13.107288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:13.108210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:13.119074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:13.145136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:13.149124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:13.227521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:13.230444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:13.230807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:13.231029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:13.236884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:13.274228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:13.284450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 822/1651] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:13.301101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:13.307205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:13.343075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:13.344548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:13.356915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 830/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:13 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:13.448324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0091726-a851-4506-b84a-6279c1a7f2ac None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:13.452750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:13.453491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:13.453834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:13.454465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:13.466991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:13.469340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:13.499768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 823/1653] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:13.964380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:13.996277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.034575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31630-6ed3-4e89-b939-a18d7451163d service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.043554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.044594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.045244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.045659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] PATH_INFO: `/v3/users/71e3deb691c24bc6927d09f38aa26d64` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.048601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e3deb691c24bc6927d09f38aa26d64)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.052902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: WARNING py.warnings [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:14.052902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: warnings.warn(msg) Sep 13 19:58:14.052902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: Sep 13 19:58:14.055549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.158205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.160551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.182104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b866-c61d-463e-8a20-538c91c576a0 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.204614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.205293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.205579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.205845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] PATH_INFO: `/v3/projects/4a5ad97df0834dc796dcfab5005f1918` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.207036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5ad97df0834dc796dcfab5005f1918)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.208051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: WARNING py.warnings [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:58:14.208051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: warnings.warn(msg) Sep 13 19:58:14.208051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: Sep 13 19:58:14.209280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.225199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e3deb691c24bc6927d09f38aa26d64'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.242916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e3deb691c24bc6927d09f38aa26d64'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.275827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e3deb691c24bc6927d09f38aa26d64'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.276986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e3deb691c24bc6927d09f38aa26d64'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.315593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee371622-8a1e-43e2-979e-1eb663c15e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e3deb691c24bc6927d09f38aa26d64'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.326111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 831/1654] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:13 2022] DELETE /identity/v3/users/71e3deb691c24bc6927d09f38aa26d64 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:14.327862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.328259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.328638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.338636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5ad97df0834dc796dcfab5005f1918', 'request_id': 'req-eb534aa8-e9ee-4187-9e8f-a32b0a576541'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.365786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5ad97df0834dc796dcfab5005f1918', 'request_id': 'req-eb534aa8-e9ee-4187-9e8f-a32b0a576541'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.367415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:14.370855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.387161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5ad97df0834dc796dcfab5005f1918', 'request_id': 'req-eb534aa8-e9ee-4187-9e8f-a32b0a576541'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.425105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] The token cache is being invalidate because project 4a5ad97df0834dc796dcfab5005f1918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:14.425421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.426511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.429068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 824/1655] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:14 2022] DELETE /identity/v3/projects/4a5ad97df0834dc796dcfab5005f1918 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:14.433146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.435590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.466439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 832/1656] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:14.466439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.466439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.508340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:14.532019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:14.589684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9048b54-6751-427b-89c2-94a0b65dd25b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.597510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb5982aa7d14e0088c329dc5a65c628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1cf0736b8a54e31ba478c01d4991716', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.598290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.598688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.599032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.600020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:14.600397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.603013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.623532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 833/1657] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:14.623532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.625342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.636871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb534aa8-e9ee-4187-9e8f-a32b0a576541 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.650005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.651697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.652312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.652816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] PATH_INFO: `/v3/users/68ab3bd47e494b2697be2d29ae3c369a` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.655093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ab3bd47e494b2697be2d29ae3c369a)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.663277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.782115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde22ac6-e7ac-4e13-86d0-4599c792e95f tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.789396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.790725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.791150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.791492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.822656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.826761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.851288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 834/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:14.861052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.871720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ab3bd47e494b2697be2d29ae3c369a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.882234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310c1c9e-9394-4967-9763-6ddf3133583b service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.893164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb5982aa7d14e0088c329dc5a65c628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1cf0736b8a54e31ba478c01d4991716', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.895741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.896195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.896666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.898578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.899318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ab3bd47e494b2697be2d29ae3c369a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.905394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.925786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ab3bd47e494b2697be2d29ae3c369a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.933527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ab3bd47e494b2697be2d29ae3c369a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.950269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ab3bd47e494b2697be2d29ae3c369a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:14.983628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 835/1659] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:14 2022] POST /identity/v3/projects => generated 354 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:14.984689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 825/1660] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:14 2022] DELETE /identity/v3/users/68ab3bd47e494b2697be2d29ae3c369a => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:14.988104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.990101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:15.116443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908433c5-ddf1-441d-bbd9-ab0694bddcd9 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:15.186328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:15.207219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb5982aa7d14e0088c329dc5a65c628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1cf0736b8a54e31ba478c01d4991716', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:15.207923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:15.208210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:15.208662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:15.209501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:15.209965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:15.212650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:15.235064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 836/1661] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:15.295306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:15.316479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721dbd7-425b-4e66-b227-96eb0cddf0a2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:15.322088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb5982aa7d14e0088c329dc5a65c628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1cf0736b8a54e31ba478c01d4991716', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:15.323067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:15.323067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:15.323366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] PATH_INFO: `/v3/projects/3a82c94bf5fb4325b0755c55d53a79ea` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:15.331322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3a82c94bf5fb4325b0755c55d53a79ea)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:15.333294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:15.409738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a82c94bf5fb4325b0755c55d53a79ea', 'request_id': 'req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:15.418001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a82c94bf5fb4325b0755c55d53a79ea', 'request_id': 'req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:15.426297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a82c94bf5fb4325b0755c55d53a79ea', 'request_id': 'req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:15.449540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] The token cache is being invalidate because project 3a82c94bf5fb4325b0755c55d53a79ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:15.450076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:15.451458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:15.460685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 826/1662] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:15 2022] DELETE /identity/v3/projects/3a82c94bf5fb4325b0755c55d53a79ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:15.528639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:15.534593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:15.685433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e91ab6-d601-487c-b3bd-e8a4c68b920b tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:15.708264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:15.709865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:15.777862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:15.849407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:15.853236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:15.855196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:15.857100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] PATH_INFO: `/v3/projects/d7bedd256d43440b9733d405dd98ad03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:15.860302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bedd256d43440b9733d405dd98ad03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:15.867524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:15.880307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ed0d8-17e9-4fcb-a0cf-52ad4af203f2 tempest-VolumeQuotaClassesTest-381448961 tempest-VolumeQuotaClassesTest-381448961-project-admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:15.917194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:15.917909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:15.918777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:15.918954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] PATH_INFO: `/v3/projects/1742a0542ad242b8812f7acb59e27c8b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:15.931606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1742a0542ad242b8812f7acb59e27c8b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:15.933876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:16.036561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bedd256d43440b9733d405dd98ad03', 'request_id': 'req-65c33136-a41c-4771-84bc-0ef91aa86250'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.062722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bedd256d43440b9733d405dd98ad03', 'request_id': 'req-65c33136-a41c-4771-84bc-0ef91aa86250'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.091612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bedd256d43440b9733d405dd98ad03', 'request_id': 'req-65c33136-a41c-4771-84bc-0ef91aa86250'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.095433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1742a0542ad242b8812f7acb59e27c8b', 'request_id': 'req-2d24e826-0490-44d6-ac6d-b1826ad59657'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.110042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1742a0542ad242b8812f7acb59e27c8b', 'request_id': 'req-2d24e826-0490-44d6-ac6d-b1826ad59657'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.118561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] The token cache is being invalidate because project d7bedd256d43440b9733d405dd98ad03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:16.120779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.122439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.124988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1742a0542ad242b8812f7acb59e27c8b', 'request_id': 'req-2d24e826-0490-44d6-ac6d-b1826ad59657'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.130588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 837/1663] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:15 2022] DELETE /identity/v3/projects/d7bedd256d43440b9733d405dd98ad03 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:16.131665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:16.133602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:16.209727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] The token cache is being invalidate because project 1742a0542ad242b8812f7acb59e27c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:16.211344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.219916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.225743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 827/1664] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:15 2022] DELETE /identity/v3/projects/1742a0542ad242b8812f7acb59e27c8b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:16.227461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:16.229081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:16.246962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:16.380535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c33136-a41c-4771-84bc-0ef91aa86250 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:16.440542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24e826-0490-44d6-ac6d-b1826ad59657 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:16.467645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:16.468634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:16.469615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:16.470223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] PATH_INFO: `/v3/users/f54f79e394174315a1f0df895d888364` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:16.473138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54f79e394174315a1f0df895d888364)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:16.477554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:16.555754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:16.556447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:16.557859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:16.558169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] PATH_INFO: `/v3/users/3bedf6fd67ce4df9845597cdde102ee4` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:16.560508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bedf6fd67ce4df9845597cdde102ee4)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:16.591297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:16.638088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54f79e394174315a1f0df895d888364'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.649516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54f79e394174315a1f0df895d888364'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.697884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:16.708581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54f79e394174315a1f0df895d888364'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.714232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54f79e394174315a1f0df895d888364'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.731976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54f79e394174315a1f0df895d888364'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.748885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 828/1665] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:16 2022] DELETE /identity/v3/users/f54f79e394174315a1f0df895d888364 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:16.749563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:16.754008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bedf6fd67ce4df9845597cdde102ee4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.757468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:16.775411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bedf6fd67ce4df9845597cdde102ee4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.828256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bedf6fd67ce4df9845597cdde102ee4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.833460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bedf6fd67ce4df9845597cdde102ee4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.872536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d20f23ed-219b-4a10-8e50-2d46a8e6d8a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bedf6fd67ce4df9845597cdde102ee4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:16.886320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 838/1666] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:16 2022] DELETE /identity/v3/users/3bedf6fd67ce4df9845597cdde102ee4 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:16.888010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1b2e-2c27-4cbf-916e-48a4b14a034a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:16.888440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1b2e-2c27-4cbf-916e-48a4b14a034a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:16.893375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f1b2e-2c27-4cbf-916e-48a4b14a034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:16.969087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ea0f1b2e-2c27-4cbf-916e-48a4b14a034a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:16.970502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ea0f1b2e-2c27-4cbf-916e-48a4b14a034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.013197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1c8ef-94af-4877-a24e-5179a6e249cb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:17.112566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:17.160641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 839/1667] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:17.160641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176235de-ffb4-4a87-96f8-d97e541263f0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.160641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176235de-ffb4-4a87-96f8-d97e541263f0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.160641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176235de-ffb4-4a87-96f8-d97e541263f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.166644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:17.167415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.168646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.168924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] PATH_INFO: `/v3/users/b22bfce9baf64a24bcfa033f55095a39` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.172588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22bfce9baf64a24bcfa033f55095a39)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:17.176686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:17.206927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-176235de-ffb4-4a87-96f8-d97e541263f0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:17.208490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-176235de-ffb4-4a87-96f8-d97e541263f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.242705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 840/1668] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:17.243783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c88ba-7c84-4140-a761-2989498f96a9 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.244080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c88ba-7c84-4140-a761-2989498f96a9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.244343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c88ba-7c84-4140-a761-2989498f96a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.278425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-162c88ba-7c84-4140-a761-2989498f96a9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:17.279692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-162c88ba-7c84-4140-a761-2989498f96a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.279962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22bfce9baf64a24bcfa033f55095a39'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:17.289089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 841/1669] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:17.291884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22bfce9baf64a24bcfa033f55095a39'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:17.292122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.292397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.292687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.317961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:17.319295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.327899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22bfce9baf64a24bcfa033f55095a39'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:17.328917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22bfce9baf64a24bcfa033f55095a39'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:17.347390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22bfce9baf64a24bcfa033f55095a39'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:17.354912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 842/1670] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:17.355874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:17.362647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.363548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 829/1671] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:16 2022] DELETE /identity/v3/users/b22bfce9baf64a24bcfa033f55095a39 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:17.367574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:17.370130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.402215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:17.528926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702f4e-f7cd-4038-b8f6-6f1126260a99 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:17.535347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1558b66-f5bf-4ea0-898c-cbc63771b44c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:17.541603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:17.544817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.545295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.545732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.546514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:17.547238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.549844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.550227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.551150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.555517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:17.671855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:17.677156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:17.700890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:17.702508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 843/1672] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:17.704428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:17.706173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:17.726832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 830/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:17.733520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6044-77ec-4bc8-9189-084aa296cd76 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:17.744578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:17.746010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.746302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.746605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.747280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:17.747766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:17.753534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:17.790242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 844/1674] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:17.794828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:17.807265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bc67a-de69-4a8e-8745-702d22926cdc service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:17.820923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:17.822009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:17.822259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:17.822714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:17.823718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:17.829192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:17.880627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 831/1675] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:17 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:17.886227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:17.904111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc7ce5-35c8-479a-8400-d5099e2f283f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:18.061239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:18.062072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.062575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.076541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.078882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:18.086058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:18.105049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:18.146272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 845/1676] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:17 2022] POST /identity/v3/users => generated 284 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:18.146958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:18.162574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533a3bd-800b-4793-8269-1bc004b942a9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:18.180685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:18.181452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.181806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.182097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.183188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:18.185139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:18.217977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 832/1677] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:18.220233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:18.248178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de298dc8-13bb-4f52-8b3a-180e7c55af16 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:18.261027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:18.264344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.264344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.266332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.267573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:18.271544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ecb32-b49d-4e15-9d08-642ff05a70d7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:18.304674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 846/1678] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:18.309097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:18.321933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee422-dc4f-45fe-b23d-adf846eeeac1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:18.340132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:18.340946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.341255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.341580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] PATH_INFO: `/v3/projects/0e3f2238a167469892edf43e7c0dc46c/users/288581ef71e5414c8b90d31f0c1bff16/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.362696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3f2238a167469892edf43e7c0dc46c, user_id=288581ef71e5414c8b90d31f0c1bff16, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:18.366006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:18.388586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 833/1679] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:18 2022] PUT /identity/v3/projects/0e3f2238a167469892edf43e7c0dc46c/users/288581ef71e5414c8b90d31f0c1bff16/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:18.401092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.401442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.401715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.453450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] MFA Rules not processed for user `288581ef71e5414c8b90d31f0c1bff16`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:18.455020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:18.699303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 847/1680] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:18.704099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:18.905991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab7d0b-461b-4d90-9ede-670eec339160 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:18.927318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:18.928254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:18.929883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:18.931667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:18.933630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:18.936758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:18.952965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:19.008903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 834/1681] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:18 2022] POST /identity/v3/projects => generated 365 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:19.013154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.031592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977b3f-032e-4f2a-bc60-8b111c26cc68 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.194320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.202516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.203232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.203649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.204883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.217106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.218136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.219791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:19.246570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2f986-0eee-49bf-938d-dad3c29f9a94 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.265525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.266216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.266535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.266815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] PATH_INFO: `/v3/projects/36a9fc0618bd410483d03d26b4ce7e27` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.270848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a9fc0618bd410483d03d26b4ce7e27)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.278243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.311366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 848/1682] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:19 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:19.316159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.346219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a0d15-2a80-46af-a674-9e853177d94a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.368214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.370015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.370307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.372393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.376152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.391029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.453051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 849/1683] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:19.462575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.488346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a9fc0618bd410483d03d26b4ce7e27', 'request_id': 'req-7a5f232b-6aa5-45a3-95e1-11532812753a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.508904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53510f5-23e8-4643-9968-024032453374 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.513563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a9fc0618bd410483d03d26b4ce7e27', 'request_id': 'req-7a5f232b-6aa5-45a3-95e1-11532812753a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.568939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a9fc0618bd410483d03d26b4ce7e27', 'request_id': 'req-7a5f232b-6aa5-45a3-95e1-11532812753a'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.597069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] The token cache is being invalidate because project 36a9fc0618bd410483d03d26b4ce7e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:19.597303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.599672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.607087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 835/1684] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:19 2022] DELETE /identity/v3/projects/36a9fc0618bd410483d03d26b4ce7e27 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:19.608038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.623327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:19.697733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:19.724867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.725579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.725860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.732226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] PATH_INFO: `/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1/users/d2c633e0fde344de9b43cd0017f9bf3c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.762643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de1b0eebe364e5ba03b28ed17aac2c1, user_id=d2c633e0fde344de9b43cd0017f9bf3c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.767388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.792401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f232b-6aa5-45a3-95e1-11532812753a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.795790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 850/1685] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:19 2022] PUT /identity/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1/users/d2c633e0fde344de9b43cd0017f9bf3c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:19.799504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.802213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:19.921420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.922219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.922612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.922957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] PATH_INFO: `/v3/users/dfb5982aa7d14e0088c329dc5a65c628` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.930979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb5982aa7d14e0088c329dc5a65c628)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.935850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.997047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:19.999821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3ed31-e8a1-440d-967e-1296e43f82b3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.016685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.019640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.019947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.022671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] PATH_INFO: `/v3/projects/5bedf7fd94364396b1984d248a753aa2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.025916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bedf7fd94364396b1984d248a753aa2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.037627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.048673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb5982aa7d14e0088c329dc5a65c628'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.069890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb5982aa7d14e0088c329dc5a65c628'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.103289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb5982aa7d14e0088c329dc5a65c628'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.104248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb5982aa7d14e0088c329dc5a65c628'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.137156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb5982aa7d14e0088c329dc5a65c628'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.159000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 836/1686] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:19 2022] DELETE /identity/v3/users/dfb5982aa7d14e0088c329dc5a65c628 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.160007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.172417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.232411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bedf7fd94364396b1984d248a753aa2', 'request_id': 'req-380629cf-c9e2-4081-a9f6-4897a05ee4b0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.259506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bedf7fd94364396b1984d248a753aa2', 'request_id': 'req-380629cf-c9e2-4081-a9f6-4897a05ee4b0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.277649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bedf7fd94364396b1984d248a753aa2', 'request_id': 'req-380629cf-c9e2-4081-a9f6-4897a05ee4b0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.299539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] The token cache is being invalidate because project 5bedf7fd94364396b1984d248a753aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:20.299877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.300664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.304157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 851/1687] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:19 2022] DELETE /identity/v3/projects/5bedf7fd94364396b1984d248a753aa2 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.305058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.308378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.329163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee914-389a-4a44-87f0-6295fe07c844 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.340433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.343269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.344024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.344820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.346344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.348706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c542a4-95d8-42b9-a397-ec2291f6ba31 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.385051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 837/1688] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:20.386638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.387505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.387915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.437645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:20.440764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.470892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 838/1689] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:20.471995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.477630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.492374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380629cf-c9e2-4081-a9f6-4897a05ee4b0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.505078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af036ffb-b3e9-4855-bd01-87ef78db092b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.506198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.507492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.508698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.509537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] PATH_INFO: `/v3/projects/f483c5fd42bd4655bc9a2bbeb49efb1a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.512123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f483c5fd42bd4655bc9a2bbeb49efb1a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.515103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.519373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.520610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.520746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.521025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] PATH_INFO: `/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1/users/d2c633e0fde344de9b43cd0017f9bf3c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.524243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:20.524751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de1b0eebe364e5ba03b28ed17aac2c1, user_id=d2c633e0fde344de9b43cd0017f9bf3c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.533415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ba289-e216-4a19-8f01-f01dfb8c4079 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.561446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 839/1690] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:20 2022] PUT /identity/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1/users/d2c633e0fde344de9b43cd0017f9bf3c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.570949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb53e-dd2b-4ece-a09b-1c293c76b220 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.570949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb53e-dd2b-4ece-a09b-1c293c76b220 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.570949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bb53e-dd2b-4ece-a09b-1c293c76b220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.649810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9c1bb53e-dd2b-4ece-a09b-1c293c76b220 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:20.658050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9c1bb53e-dd2b-4ece-a09b-1c293c76b220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.690221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f483c5fd42bd4655bc9a2bbeb49efb1a', 'request_id': 'req-81808ff0-7f7f-474b-bda4-af0969e7469e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.714621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f483c5fd42bd4655bc9a2bbeb49efb1a', 'request_id': 'req-81808ff0-7f7f-474b-bda4-af0969e7469e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.757145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f483c5fd42bd4655bc9a2bbeb49efb1a', 'request_id': 'req-81808ff0-7f7f-474b-bda4-af0969e7469e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.802739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] The token cache is being invalidate because project f483c5fd42bd4655bc9a2bbeb49efb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:20.803053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.807714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-81808ff0-7f7f-474b-bda4-af0969e7469e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.810985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 852/1691] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:20 2022] DELETE /identity/v3/projects/f483c5fd42bd4655bc9a2bbeb49efb1a => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.819583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.821271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.821586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.890820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 840/1692] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:20.892438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.893170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.893938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.898228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] MFA Rules not processed for user `d2c633e0fde344de9b43cd0017f9bf3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:20.899576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.961429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:20.963031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.015981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 841/1693] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:21.015981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.074810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db068c0-9f04-48ac-a0bd-26e78dfb5646 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.109956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.114811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.115396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.115812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.118092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:21.118973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.125686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d332730-7de8-48c8-9cf5-ef33adaa9803 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.129319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:21.162300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 842/1694] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:21.166477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.167258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.169334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.194678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 853/1695] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:21.195935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.217927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:21.220085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.223590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a2c3e-3f15-40b8-9126-54b33c7cd695 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.233276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.236562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.238685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.239234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.240150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.242728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.246899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 843/1696] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:21.248367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.252360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.329809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 854/1697] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:21.330934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.422753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855da813-8707-4cae-b9c9-9a7a035352ed None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.467840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:21.477979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a72971-0292-43b8-ad61-4e428b7c4524 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.498682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.499588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.500263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.500698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.503533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:21.504917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.519650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.521368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.522219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.522708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.523342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.551311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 855/1698] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:21.552029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.572779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1d9b5-cd15-4260-b7ed-5971d61024d1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.592666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.593848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.594133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.594432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.595103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.597177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d762-fdb5-4f58-bf06-bb5055973c79 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.645864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 856/1699] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:21.647531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.647965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.648360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.701775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.705427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.721871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] MFA Rules not processed for user `bd55568bf88346b794922ef8555f9654`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:21.723519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.727919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 844/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:21.729421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.747920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfe8aa-d8ec-45f3-951f-28dc8fa6479d service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.754262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.755172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.755556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.755931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.757367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.759657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.851028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 845/1701] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:21.854299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.901476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 857/1702] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:21.902769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.934059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f40c7c-9a72-4d68-b5ea-48095325eee6 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.987056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:22.048996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb707a-43c4-4ba6-bb9c-22a8af0b2bb9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.057258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.061167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.061715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.062182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.063773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.072399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572942b-97a1-49e6-a79d-48fc97c1cf5a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.098642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 846/1703] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:22.100325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.100651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.101083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.113778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.114456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.114737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.115006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.115654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.118082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.149170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] MFA Rules not processed for user `bd55568bf88346b794922ef8555f9654`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:22.151216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.164497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 858/1704] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/users => generated 302 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:22.164497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.169437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.188872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d963ea-f2e1-478b-803d-42acf45a606c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.198310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.201615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.202002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.202496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] PATH_INFO: `/v3/users/848f3bab2f8b404aaf6eca4beeb78487` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.205006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=848f3bab2f8b404aaf6eca4beeb78487)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.208329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.297121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 847/1705] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:22.301650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.304238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.326460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:22.331387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848f3bab2f8b404aaf6eca4beeb78487'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.349735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848f3bab2f8b404aaf6eca4beeb78487'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.387929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848f3bab2f8b404aaf6eca4beeb78487'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.391604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848f3bab2f8b404aaf6eca4beeb78487'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.414800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848f3bab2f8b404aaf6eca4beeb78487'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.445954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 859/1706] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:22 2022] DELETE /identity/v3/users/848f3bab2f8b404aaf6eca4beeb78487 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:22.445954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.445954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.455620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191d411-9afe-41ca-9a6b-9d858388cf8d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.557336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.558632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.559066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.559382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.563356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.611772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef295-adb9-499b-8f43-47622beabc6f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.633520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.634650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.635132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.636278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] PATH_INFO: `/v3/projects/c1cf0736b8a54e31ba478c01d4991716` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.637507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1cf0736b8a54e31ba478c01d4991716)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.643420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.660114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:22.695247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.698576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.733565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 848/1707] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:22.740297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.750347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.751449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1cf0736b8a54e31ba478c01d4991716', 'request_id': 'req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.771493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1cf0736b8a54e31ba478c01d4991716', 'request_id': 'req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.789438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1cf0736b8a54e31ba478c01d4991716', 'request_id': 'req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.792725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482f7e-017e-49a7-94f5-001071e4c2e2 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.817394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] The token cache is being invalidate because project c1cf0736b8a54e31ba478c01d4991716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:22.817394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.819533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:22.825950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 860/1708] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:22 2022] DELETE /identity/v3/projects/c1cf0736b8a54e31ba478c01d4991716 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:22.825950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.826195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.937312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.938457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.939064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.939538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.940620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.944284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.977209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:22.980096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 849/1709] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:22.987497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.991959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.011946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795be9-a4b7-4d6c-bf4f-77a1cbc757ca admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.028663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.029641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.029989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.030399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.031154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.037283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe35f75-8765-4de9-b8d6-99d7c7eddff9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.063025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 861/1710] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.068702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25364886-4ad3-435b-96fd-9ffddf39b159 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.071650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25364886-4ad3-435b-96fd-9ffddf39b159 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.072206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25364886-4ad3-435b-96fd-9ffddf39b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.130947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-25364886-4ad3-435b-96fd-9ffddf39b159 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:23.133849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-25364886-4ad3-435b-96fd-9ffddf39b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.165075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 862/1711] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:23.166696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.167152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.167477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.216195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:23.220816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.230752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec621db4-33aa-43eb-b772-e0c7de72a14e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.239555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.240313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.240593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.240867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.242314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 863/1712] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:23.244625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.247280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.253314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.282633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e26f5-c0a3-46d8-96dd-0a9a9d480dcb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.298805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.299874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.300358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.300876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] PATH_INFO: `/v3/projects/c290998a5b844aa8bd4564d166a78608/users/51afa520aa3d40a8a6bcd9aedab60454/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.330682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290998a5b844aa8bd4564d166a78608, user_id=51afa520aa3d40a8a6bcd9aedab60454, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.334238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.378449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 864/1713] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:23 2022] PUT /identity/v3/projects/c290998a5b844aa8bd4564d166a78608/users/51afa520aa3d40a8a6bcd9aedab60454/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:23.381464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.444409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.446567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.471768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 850/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:23.484100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.494220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.543069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:23.584513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890df547-e44b-4298-a0d6-4282e7438df1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.599423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.600773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.603224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.603224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.603589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.606538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65173c-fbab-47e6-a679-d529a8bfc085 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.636971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 865/1715] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.641119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060b6a-5d69-406c-a196-af6641903e1c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.641756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060b6a-5d69-406c-a196-af6641903e1c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.642132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d060b6a-5d69-406c-a196-af6641903e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.649894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fda61-1ce8-44bc-bd4d-68085ab37978 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.689334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-1d060b6a-5d69-406c-a196-af6641903e1c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:23.692507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1d060b6a-5d69-406c-a196-af6641903e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.725230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 866/1716] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:23.727591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5737b-c835-4bfd-8a7d-6654768dfe4e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.727994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5737b-c835-4bfd-8a7d-6654768dfe4e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.728859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5737b-c835-4bfd-8a7d-6654768dfe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.764952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.765611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.766834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.767115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.794544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.796715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.819478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 851/1717] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:23.821239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.834580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f4a5737b-c835-4bfd-8a7d-6654768dfe4e None None] MFA Rules not processed for user `51afa520aa3d40a8a6bcd9aedab60454`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:23.837570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f4a5737b-c835-4bfd-8a7d-6654768dfe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.848312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e4ba-0f49-499d-ac98-ab1ab2927431 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.859999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.860826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.862176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.862719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] PATH_INFO: `/v3/projects/7cef6e663f8849e4bbff2bc7a77f11e6/users/f6ec10e0134641468b49558d9fb1cacc/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.900074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cef6e663f8849e4bbff2bc7a77f11e6, user_id=f6ec10e0134641468b49558d9fb1cacc, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.904294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.930513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 852/1718] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:23 2022] PUT /identity/v3/projects/7cef6e663f8849e4bbff2bc7a77f11e6/users/f6ec10e0134641468b49558d9fb1cacc/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:23.931573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.012820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 867/1719] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:24.014764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.015317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.016353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.063267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:24.066717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:24.123938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5ff45-4b91-49ae-9396-21ca6ef300e8 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.141655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.142806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.143276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.143833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.145470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:24.147068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:24.147536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.150422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.168861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 853/1720] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:24.170184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.188762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adf632-b414-49f4-97ae-f1707fded453 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.197792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.198473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.198756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.199027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.199673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.205447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.209647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 868/1721] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:24.210846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.234280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e1aa-aee7-41c2-b823-67c65a76117e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.249351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.250871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.251620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.252379 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.253512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.256329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.271442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 854/1722] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:24.272596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.311323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 869/1723] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:24.315303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.328198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:24.414120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349de6c-ae69-4efa-befc-ed27b916ec74 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.448488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddaae3-6a6b-4330-9e69-a330b6fa283c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.464324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.465699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.466178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.466780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.470214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.472956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.523200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 870/1724] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:24.523200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.525483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:24.548953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.549880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.550867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.550987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.553359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.556090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.602418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 855/1725] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/users => generated 292 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:24.602418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.613013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6de43-2ee0-4d75-9683-2f8f5ea423c4 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.621233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.622041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.625335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.625335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.625335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.625701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.639168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37871b-2a30-4f19-a4f5-20ad8c9a6890 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.644078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 856/1726] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:24.645205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.649928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.650621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.650907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.651177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.667615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5e2bf-028f-4704-89bd-25c718fcdf21 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.674862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.675669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.676102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.676477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.677244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.679573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.695955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.698442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.716188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 857/1727] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:24.717367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.726719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 871/1728] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:24.727597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.732929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fb046-5243-41bb-97fe-4f59cec3e386 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.743631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e4392-1651-4458-ac7d-1d1252253491 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.744650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.745967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.745967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.746271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.747139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.750422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.756787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.757083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.757362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.757634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] PATH_INFO: `/v3/projects/8c124adab5ad41c4ade7b4bf37052c17/users/0206a0a96c614028bfd2e47fa5b69b9a/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.773249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 858/1729] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:24.775720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.798851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c124adab5ad41c4ade7b4bf37052c17, user_id=0206a0a96c614028bfd2e47fa5b69b9a, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.811887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4766cc3-2712-4b47-b087-812ce3a3776c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.831270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613f1f3-f637-4eea-a963-b340a3fc89a7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.852289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.854013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.854739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.859935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] PATH_INFO: `/v3/projects/742c8612be11476f92b38683ccd818bd/users/dbb6467774bf4fb3ae9936327a58da65/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.869932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 872/1730] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:24 2022] PUT /identity/v3/projects/8c124adab5ad41c4ade7b4bf37052c17/users/0206a0a96c614028bfd2e47fa5b69b9a/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:24.881553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942cd7-56e8-4dd4-ab79-68c3160d5aea None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.881844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942cd7-56e8-4dd4-ab79-68c3160d5aea None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.882124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84942cd7-56e8-4dd4-ab79-68c3160d5aea None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.890626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 873/1731] 10.222.0.52 () {54 vars in 912 bytes} [Tue Sep 13 19:58:24 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:24.893158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84942cd7-56e8-4dd4-ab79-68c3160d5aea None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.931232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] RBAC: Authorizing `identity:create_grant(project_id=742c8612be11476f92b38683ccd818bd, user_id=dbb6467774bf4fb3ae9936327a58da65, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.936893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60574a81-58c6-4fb1-aec8-52a1affb755f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.969323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:24.971697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 859/1732] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:24 2022] PUT /identity/v3/projects/742c8612be11476f92b38683ccd818bd/users/dbb6467774bf4fb3ae9936327a58da65/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:24.975070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6050-990b-4166-b906-600c952d3425 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.976244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6050-990b-4166-b906-600c952d3425 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.976444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6050-990b-4166-b906-600c952d3425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.114972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ba1e6050-990b-4166-b906-600c952d3425 None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:25.118247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ba1e6050-990b-4166-b906-600c952d3425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.147477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84942cd7-56e8-4dd4-ab79-68c3160d5aea None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:25.257562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:25.265559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 860/1733] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:58:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:25.268238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.268893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.269738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.307181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:25.308021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.308128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.308350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.309002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:25.310728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:25.328204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 874/1734] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:25.335502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] MFA Rules not processed for user `dbb6467774bf4fb3ae9936327a58da65`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:25.335661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:25.337253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.342398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.367533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55030ca4-838c-476e-95f5-8d73bf5bc011 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:25.373055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:25.373821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.374158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.374464 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] PATH_INFO: `/v3/projects/3fdc25924a1446f883d7993a9d4cc5b9` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.375794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fdc25924a1446f883d7993a9d4cc5b9)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:25.379380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:25.447129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 861/1735] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:25.455616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:25.463142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fdc25924a1446f883d7993a9d4cc5b9', 'request_id': 'req-41a9db2a-be2f-4a87-bc3d-90889a57d381'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:25.469743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fdc25924a1446f883d7993a9d4cc5b9', 'request_id': 'req-41a9db2a-be2f-4a87-bc3d-90889a57d381'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:25.476697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df9241-206e-4d6b-9693-107c8b449c57 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:25.477987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fdc25924a1446f883d7993a9d4cc5b9', 'request_id': 'req-41a9db2a-be2f-4a87-bc3d-90889a57d381'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:25.497376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] The token cache is being invalidate because project 3fdc25924a1446f883d7993a9d4cc5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:25.497676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:25.498577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:25.503857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 875/1736] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:25 2022] DELETE /identity/v3/projects/3fdc25924a1446f883d7993a9d4cc5b9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:25.503857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:25.503857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.585862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:25.592052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:25.601534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.602242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.602677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.605110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.620365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:25.622010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9db2a-be2f-4a87-bc3d-90889a57d381 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:25.626230 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:25.645089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 862/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:25.646217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:25.647785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.667176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b8931-fdd0-46e3-b3c5-6f8ce331c67e service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:25.671582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:25.672486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.672768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.672872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:25.673041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] PATH_INFO: `/v3/projects/8c124adab5ad41c4ade7b4bf37052c17/users/0206a0a96c614028bfd2e47fa5b69b9a/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.673841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.674171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.674678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.675639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:25.679769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357b73b-476a-4bbf-9654-c3d393499fa3 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:25.682855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c124adab5ad41c4ade7b4bf37052c17, user_id=0206a0a96c614028bfd2e47fa5b69b9a, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:25.696071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef0527-6dc7-4645-acd8-bbe462e3e7dd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:25.730930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 876/1738] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:25 2022] PUT /identity/v3/projects/8c124adab5ad41c4ade7b4bf37052c17/users/0206a0a96c614028bfd2e47fa5b69b9a/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:25.745722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.746612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.749198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.751395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 863/1739] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:25 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:25.755345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126015d-cfa1-42ba-8a03-a9c5a72da2bd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:25.755873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126015d-cfa1-42ba-8a03-a9c5a72da2bd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:25.757619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126015d-cfa1-42ba-8a03-a9c5a72da2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:25.807666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:25.811397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:25.836766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d126015d-cfa1-42ba-8a03-a9c5a72da2bd None None] MFA Rules not processed for user `0206a0a96c614028bfd2e47fa5b69b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:25.839906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d126015d-cfa1-42ba-8a03-a9c5a72da2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:26.043306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 877/1740] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:26.044296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.083390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f4362-260b-4ad3-8f1d-ddd6d171068c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.084647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:26.099015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.099686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.099972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.100249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.101155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.106225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.107506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 864/1741] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:26.109421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.110182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.110493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.145723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:26.147002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:26.161366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 865/1742] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:26.162107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.164691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:26.166248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 878/1743] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:26 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:26.167303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.195620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a966ef-284f-4568-96ac-307f5d986929 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.202170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.202835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.203064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.203430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.204011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:26.204321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.208591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.240902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 879/1744] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:26.241815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.267633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f3cbf-c2f2-4920-aa98-542565bd78a9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.282089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.283734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.284539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.285097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.287862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.289713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.316809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209838bb-4a5c-4d72-a3de-993e8654b1fc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.326597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 880/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:26.329023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.334180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.335456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.335456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.337531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.363054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012068f-49a1-440d-958f-e7f3e584b9db admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.380196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.381745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.386708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.388184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.389080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.391357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.392839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.395674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 866/1746] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:26.402327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.407575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.434820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd1460-dbdb-4435-a250-5b08e2e1bbd8 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.453552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.455056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.455569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.456174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] PATH_INFO: `/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce/users/aadeb71e6fc34b9a9615d4211af86109/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.505506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 881/1747] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:26 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:26.506719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.521342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ad8d8a58594c2dbfb8c17c27ed2dce, user_id=aadeb71e6fc34b9a9615d4211af86109, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.530771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.597072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 867/1748] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:26 2022] PUT /identity/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce/users/aadeb71e6fc34b9a9615d4211af86109/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:26.601621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.713005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be570-bede-4d8b-9f41-942f4f48b3bb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.787096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:26.887192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a07738-e97f-4776-b51c-edef60369647 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.893301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.895600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.896118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.896578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.897406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.899469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af2fab-aaf4-4d7f-91fc-83f18389eeca admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.914237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 868/1749] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:26.916095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.916384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.916660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.958684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.960531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.960838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.964689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.965616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.972181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1aec40-140d-4213-b68f-dd1c0b334f65 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.981193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] MFA Rules not processed for user `b07b8b251aaa456c9916c41f9a0b77e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:26.985138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.074518 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 882/1750] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:26 2022] POST /identity/v3/users => generated 283 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:27.076749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.077218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.077747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.128206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:27.132323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] MFA Rules not processed for user `2577fc11946c4597857187ae7b464fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:27.134118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.180366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 869/1751] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:27.181551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:27.187897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.378433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09c483-a14f-4813-aa45-7e49aa671609 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:27.380233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 883/1752] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:27.381516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:27.397123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:27.398082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.398591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.399047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.402164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.402291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba0e1-161b-4900-bc77-8a6bf3998426 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:27.407399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:27.408165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.408474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.408750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] PATH_INFO: `/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce/users/aadeb71e6fc34b9a9615d4211af86109/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.410709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ad8d8a58594c2dbfb8c17c27ed2dce, user_id=aadeb71e6fc34b9a9615d4211af86109, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:27.416295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:27.436458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:27.439351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:27.441241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 884/1753] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:27 2022] PUT /identity/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce/users/aadeb71e6fc34b9a9615d4211af86109/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:27.441865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:27.462286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 870/1754] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:27.463463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:27.466145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.633619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9981f-0a68-4426-8673-38c37ee85421 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:27.660928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:27.661473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.661894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.662158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.662888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:27.666312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649f60-3740-4e78-ba02-ae579444b54c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:27.683641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c115d14-8e81-4e07-97f9-017546431411 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:27.694162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 885/1755] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:27.694162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b77b-3077-4861-8fa1-2ccd9c825332 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.694162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b77b-3077-4861-8fa1-2ccd9c825332 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.694162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b77b-3077-4861-8fa1-2ccd9c825332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.742516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8829b77b-3077-4861-8fa1-2ccd9c825332 None None] MFA Rules not processed for user `2577fc11946c4597857187ae7b464fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:27.746355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8829b77b-3077-4861-8fa1-2ccd9c825332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:27.797222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:27.895615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:27.898761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.900052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.900540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:27.931912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:27.968061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 886/1756] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:27.970175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519a2b-79af-4ad2-82ed-74112b7ef05f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:27.970527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519a2b-79af-4ad2-82ed-74112b7ef05f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:27.970854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1519a2b-79af-4ad2-82ed-74112b7ef05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.041985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a1519a2b-79af-4ad2-82ed-74112b7ef05f None None] MFA Rules not processed for user `aadeb71e6fc34b9a9615d4211af86109`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:28.044143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a1519a2b-79af-4ad2-82ed-74112b7ef05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:28.094245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:28.097279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:28.142274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 871/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:28.143728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:28.197035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef9778-3516-428d-a4b6-873d1ca8691c service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:28.210990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:28.212548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:28.213017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:28.213473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.214633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:28.216713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:28.236781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 872/1758] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:28.240435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:28.254923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f487195-52f2-4091-a9d6-e79fa6471315 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:28.259923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:28.260596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:28.260884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:28.261176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] PATH_INFO: `/v3/users/51afa520aa3d40a8a6bcd9aedab60454` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.263370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51afa520aa3d40a8a6bcd9aedab60454)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:28.265004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:28.336996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 887/1759] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:28.349298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:28.349835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:28.350100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.433100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51afa520aa3d40a8a6bcd9aedab60454'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:28.435760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] MFA Rules not processed for user `f6ec10e0134641468b49558d9fb1cacc`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:28.437128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:28.473503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51afa520aa3d40a8a6bcd9aedab60454'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:28.526275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51afa520aa3d40a8a6bcd9aedab60454'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:28.528789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51afa520aa3d40a8a6bcd9aedab60454'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:28.560234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51afa520aa3d40a8a6bcd9aedab60454'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:28.584245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 873/1760] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:28 2022] DELETE /identity/v3/users/51afa520aa3d40a8a6bcd9aedab60454 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:28.584245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:28.586005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:28.637890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 888/1761] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:28.638891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:28.643361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:28.697658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6258e-40cd-4244-82fa-75847d2c7e1d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:28.828153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc157-bc54-492d-bc09-0589512e6c8e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:28.836480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:28.855228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:28.856091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:28.856537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:28.857751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.870441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:28.905614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:28.922546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:28.925840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:28.931407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:28.931749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] PATH_INFO: `/v3/projects/90f39579107b4134acedd96b3dd97ecb/users/6f32985d82194bb99d0d3411d212e4ad/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:28.996905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f39579107b4134acedd96b3dd97ecb, user_id=6f32985d82194bb99d0d3411d212e4ad, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:29.004890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:29.056083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 889/1762] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:28 2022] PUT /identity/v3/projects/90f39579107b4134acedd96b3dd97ecb/users/6f32985d82194bb99d0d3411d212e4ad/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:29.058982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:29.071270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.108089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:29.109793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f195-52a7-4861-997c-ddec869fbefc service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:29.129054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 874/1763] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:29.130724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:29.130950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:29.131650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:29.198573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:29.201179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.256295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b46da-514a-424c-98a3-71adf29e0aa9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:29.270748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:29.273025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:29.274021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:29.278887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:29.283451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.365006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 875/1764] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:29.376026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:29.382999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.415652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d6181-b185-4e59-873d-3b53b4e89f95 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:29.430360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:29.431289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:29.431700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:29.433550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:29.438175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:29.445146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb38713-8efe-4566-9c7b-05c1340babeb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:29.492054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:29.495667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:29.522825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 876/1765] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:29 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:29.523179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 890/1766] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:29 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:29.527436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:29.528078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:29.528763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:29.541512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:29.546098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.614275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] MFA Rules not processed for user `6f32985d82194bb99d0d3411d212e4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:29.622704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:29.740614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6436-d193-44fb-bf79-52e900fb958e service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:29.826105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:29.884201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:29.884866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:29.885170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:29.885477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:29.887213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 877/1767] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:29.888456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:29.955579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:30.028738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.030260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.051656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 891/1768] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:30.052365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.053744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:30.066047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc4f4-b192-4aa4-a167-963f0bc59636 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.084689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.085335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.085610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.085876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.086654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.088656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.131893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2b9-da6d-4291-af0c-eda416ff6d8a service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.138582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.140209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.141121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.141561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.143890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:30.144770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 878/1769] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:29 2022] POST /identity/v3/users => generated 308 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:30.145656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.171832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c90296-9802-40cd-9399-149ec4e8779b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.181052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.185264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.186949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.193916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.238480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.248063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.281194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.282754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.300982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 879/1770] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:30.302081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.328370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8475d1-99e8-46a9-8c43-a9ce4effa1ad service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.333160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 892/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:30.336349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.336986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.337315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.337756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.338634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.340954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.356800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 880/1772] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:30.361645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.381435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178c6c6-f2a6-4efe-8290-99172cb4916b service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.395802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.396505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.396782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.397069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] PATH_INFO: `/v3/projects/e668623112ac436ea9dc0da7d83c68c1/users/5874d4f28dc140fbb8fe22606318ac5d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.437723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e668623112ac436ea9dc0da7d83c68c1, user_id=5874d4f28dc140fbb8fe22606318ac5d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.440437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a7608-a5a3-4da6-8aa1-b9ddc484cea0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.469850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 893/1773] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:30 2022] PUT /identity/v3/projects/e668623112ac436ea9dc0da7d83c68c1/users/5874d4f28dc140fbb8fe22606318ac5d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:30.475615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.496244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72d3be-83b8-40ce-8f56-4d390261ebda admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.736980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.741850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.742965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.744112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.746533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.764170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb29f-8dfc-43e0-83c2-546b96aadc59 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.764715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb29f-8dfc-43e0-83c2-546b96aadc59 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.764905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccb29f-8dfc-43e0-83c2-546b96aadc59 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.766824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 894/1774] 10.222.0.52 () {54 vars in 912 bytes} [Tue Sep 13 19:58:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:30.767380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.787003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c17a2-61e8-4beb-b7b9-34d47f1da1f6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.787322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c17a2-61e8-4beb-b7b9-34d47f1da1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.787644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c17a2-61e8-4beb-b7b9-34d47f1da1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.833921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 881/1775] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:30.837636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:30.883856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8c8c17a2-61e8-4beb-b7b9-34d47f1da1f6 None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:30.889592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8c8c17a2-61e8-4beb-b7b9-34d47f1da1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:30.895514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec141f-98cb-40b0-9fef-9e9f286e4d57 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:30.916149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:30.921222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:30.922666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:30.925888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] PATH_INFO: `/v3/projects/e668623112ac436ea9dc0da7d83c68c1/users/5874d4f28dc140fbb8fe22606318ac5d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:30.934841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e668623112ac436ea9dc0da7d83c68c1, user_id=5874d4f28dc140fbb8fe22606318ac5d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:30.947529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:30.949414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:31.021662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 882/1776] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:30 2022] PUT /identity/v3/projects/e668623112ac436ea9dc0da7d83c68c1/users/5874d4f28dc140fbb8fe22606318ac5d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:31.224422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 895/1777] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:58:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:31.499754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:31.512382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:31.512679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:31.512961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:31.724747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:31.728818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef31c0-517e-4d78-ad72-d14e8d97c7e8 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:31.731909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:31.751852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:31.752115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:31.756943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:31.757301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:31.803804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:31.805193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:31.833435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 883/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:31.834676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:31.837390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:31.900734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:32.005261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 896/1779] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:32.027938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.029478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.069697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3562a8-56f7-4bbd-8589-a700014387f8 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.075598 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:32.076718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.077129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.077506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] PATH_INFO: `/v3/projects/c290998a5b844aa8bd4564d166a78608` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.081430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c290998a5b844aa8bd4564d166a78608)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:32.083966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:32.225956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78782a-0ef6-44eb-9c12-6195d3e2fcdf None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.233628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c290998a5b844aa8bd4564d166a78608', 'request_id': 'req-520633da-8c60-4d68-bdd4-9b2c85fa33e2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.249380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c290998a5b844aa8bd4564d166a78608', 'request_id': 'req-520633da-8c60-4d68-bdd4-9b2c85fa33e2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.267924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c290998a5b844aa8bd4564d166a78608', 'request_id': 'req-520633da-8c60-4d68-bdd4-9b2c85fa33e2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.312536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] The token cache is being invalidate because project c290998a5b844aa8bd4564d166a78608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:32.314195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.315896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.322737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 884/1780] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:31 2022] DELETE /identity/v3/projects/c290998a5b844aa8bd4564d166a78608 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:32.324130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.328470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.355317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:32.414738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:32.415668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.422973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.423601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] PATH_INFO: `/v3/users/288581ef71e5414c8b90d31f0c1bff16` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.427810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288581ef71e5414c8b90d31f0c1bff16)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:32.429483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:32.590853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288581ef71e5414c8b90d31f0c1bff16'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.596439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520633da-8c60-4d68-bdd4-9b2c85fa33e2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.603810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:32.605192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.605794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.606247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.609584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.610307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288581ef71e5414c8b90d31f0c1bff16'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.710692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288581ef71e5414c8b90d31f0c1bff16'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.713276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288581ef71e5414c8b90d31f0c1bff16'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.728244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:32.738465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288581ef71e5414c8b90d31f0c1bff16'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.769340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 897/1781] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:32 2022] DELETE /identity/v3/users/288581ef71e5414c8b90d31f0c1bff16 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:32.770350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.778142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.887379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:32.895002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae33ce0-eece-4c1f-bdd2-5abfa847e947 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:32.949152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 885/1782] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 19:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:58:32.977522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb4541-c27c-4b6a-ac0b-f7ba8b8db574 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.988048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.997823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.999879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.021849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:33.024704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:33.025121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:33.025419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] PATH_INFO: `/v3/users/0206a0a96c614028bfd2e47fa5b69b9a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.032734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0206a0a96c614028bfd2e47fa5b69b9a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:33.034351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:33.111183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:33.116715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:33.143419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 886/1783] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:33.174990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:33.177505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:33.297881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0206a0a96c614028bfd2e47fa5b69b9a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.341075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0206a0a96c614028bfd2e47fa5b69b9a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.380406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:33.391910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0206a0a96c614028bfd2e47fa5b69b9a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.394075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0206a0a96c614028bfd2e47fa5b69b9a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.431029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fd70e53-a223-4b3d-9328-14f1e5b43d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0206a0a96c614028bfd2e47fa5b69b9a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.455663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 898/1784] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:32 2022] DELETE /identity/v3/users/0206a0a96c614028bfd2e47fa5b69b9a => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:33.522670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac44a3-dd60-47f9-a26d-ea3295c0b157 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:33.895427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:33.896336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:33.896455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:33.896716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.902264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.198769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.200903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd51b1e-e20b-4db2-be10-16582ac2a7bc service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.215405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 887/1785] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 1042 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:34.342136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.349237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.349237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.378872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] MFA Rules not processed for user `2577fc11946c4597857187ae7b464fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:34.380501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.620591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 899/1786] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:36.179251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:36.179845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:36.179845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:36.189203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:36.191140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:36.244288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0b6e0-8586-46ca-8ab8-69951357fa3c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:36.274256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:36.275463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:36.276991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:36.277488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] PATH_INFO: `/v3/projects/0e3f2238a167469892edf43e7c0dc46c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:36.282063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3f2238a167469892edf43e7c0dc46c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:36.286447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:36.296507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] MFA Rules not processed for user `5874d4f28dc140fbb8fe22606318ac5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:36.298357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:36.329138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:36.478241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3f2238a167469892edf43e7c0dc46c', 'request_id': 'req-a77cca94-db26-4da3-b408-62599b8ed5d2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:36.512257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3f2238a167469892edf43e7c0dc46c', 'request_id': 'req-a77cca94-db26-4da3-b408-62599b8ed5d2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:36.535712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3f2238a167469892edf43e7c0dc46c', 'request_id': 'req-a77cca94-db26-4da3-b408-62599b8ed5d2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:36.599878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] The token cache is being invalidate because project 0e3f2238a167469892edf43e7c0dc46c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:36.601344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:36.609825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a77cca94-db26-4da3-b408-62599b8ed5d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:36.632588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 900/1787] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:36 2022] DELETE /identity/v3/projects/0e3f2238a167469892edf43e7c0dc46c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:36.644058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727db04-2ffc-4857-9e1f-f1f33c75b3f5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:36.651579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727db04-2ffc-4857-9e1f-f1f33c75b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:36.652034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727db04-2ffc-4857-9e1f-f1f33c75b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:36.685836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 888/1788] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:36 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:36.689065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:36.755896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf4e02-c5ed-4f59-ba5c-f4b252b7e683 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:36.784425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f727db04-2ffc-4857-9e1f-f1f33c75b3f5 None None] MFA Rules not processed for user `b07b8b251aaa456c9916c41f9a0b77e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:36.789000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f727db04-2ffc-4857-9e1f-f1f33c75b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:37.034425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:37.034425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:37.034425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:37.034425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] PATH_INFO: `/v3/users/d2c633e0fde344de9b43cd0017f9bf3c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:37.061318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c633e0fde344de9b43cd0017f9bf3c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:37.069026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:37.093931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 901/1789] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:37.099822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:37.101400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:37.102115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:37.191944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:37.199386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:37.249058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:37.298731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c633e0fde344de9b43cd0017f9bf3c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:37.343825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c633e0fde344de9b43cd0017f9bf3c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:37.410323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c633e0fde344de9b43cd0017f9bf3c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:37.411610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c633e0fde344de9b43cd0017f9bf3c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:37.449416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c633e0fde344de9b43cd0017f9bf3c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:37.457083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 902/1790] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Sep 13 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:37.460094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:37.464895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:37.520582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2c804-3656-42af-98d8-703fd2d67d15 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:37.524995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 889/1791] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:36 2022] DELETE /identity/v3/users/d2c633e0fde344de9b43cd0017f9bf3c => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:37.526251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:37.530720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:37.700274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:37.817821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:37.826390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:37.826390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:37.826390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] PATH_INFO: `/v3/projects/8c124adab5ad41c4ade7b4bf37052c17` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:37.828410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c124adab5ad41c4ade7b4bf37052c17)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:37.846122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:37.890311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ea18-dc42-4084-9d98-5cc20858f423 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:37.941073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:37.942329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:37.942863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:37.943355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.019086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c124adab5ad41c4ade7b4bf37052c17', 'request_id': 'req-3a2fcc2a-5910-4b45-b566-9353fa04dc96'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.025112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:38.028915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:38.052139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c124adab5ad41c4ade7b4bf37052c17', 'request_id': 'req-3a2fcc2a-5910-4b45-b566-9353fa04dc96'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.067913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 890/1792] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:38.068863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:38.070424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c124adab5ad41c4ade7b4bf37052c17', 'request_id': 'req-3a2fcc2a-5910-4b45-b566-9353fa04dc96'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.072918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:38.099820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] The token cache is being invalidate because project 8c124adab5ad41c4ade7b4bf37052c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:38.100124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.109692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3a2fcc2a-5910-4b45-b566-9353fa04dc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.113110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 903/1793] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:37 2022] DELETE /identity/v3/projects/8c124adab5ad41c4ade7b4bf37052c17 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:38.121085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c5f3-7454-4909-972a-61af7c1bee87 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:38.121381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c5f3-7454-4909-972a-61af7c1bee87 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:38.121653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10c5f3-7454-4909-972a-61af7c1bee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.129000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f99f68-4519-4abd-a25b-518cd91ebf29 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:38.222224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9f10c5f3-7454-4909-972a-61af7c1bee87 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:38.245769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9f10c5f3-7454-4909-972a-61af7c1bee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:38.561028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:38.561659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:38.561749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:38.561749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:38.561749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] PATH_INFO: `/v3/users/dbb6467774bf4fb3ae9936327a58da65` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.565312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb6467774bf4fb3ae9936327a58da65)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:38.570919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:38.599001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 904/1794] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:38.613303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:38.618069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:38.618465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.719998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:38.725270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:38.741680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb6467774bf4fb3ae9936327a58da65'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.758029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 905/1795] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:38.759784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:38.763833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb6467774bf4fb3ae9936327a58da65'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.785213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb6467774bf4fb3ae9936327a58da65'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.787798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb6467774bf4fb3ae9936327a58da65'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.806087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51c6a311-3d5e-42c6-a9c3-aee7ac022775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb6467774bf4fb3ae9936327a58da65'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:38.816211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9ee67-d03e-4056-8d8b-1b23e0167a02 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:38.817646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 891/1796] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:38 2022] DELETE /identity/v3/users/dbb6467774bf4fb3ae9936327a58da65 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:38.825733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:38.826020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:38.826298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.931062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:38.932394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:38.992024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:38.992818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:38.993162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:38.995018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:38.995258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:38.995610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:38.997339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.017644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 906/1797] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:39.018577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.020927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.037217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8291bd2-2cf9-445e-a7bc-29fe0c84fdac admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:39.063342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:39.067543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.068105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.068384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] PATH_INFO: `/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.069857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de1b0eebe364e5ba03b28ed17aac2c1)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:39.072194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.097473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:39.154517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 892/1798] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:39.155455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.166293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.212371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de1b0eebe364e5ba03b28ed17aac2c1', 'request_id': 'req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:39.232310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a81c5-9e12-4b63-9ee4-61521b0588cc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:39.237065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de1b0eebe364e5ba03b28ed17aac2c1', 'request_id': 'req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:39.253749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de1b0eebe364e5ba03b28ed17aac2c1', 'request_id': 'req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:39.275155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] The token cache is being invalidate because project 2de1b0eebe364e5ba03b28ed17aac2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:39.276674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:39.277455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:39.279949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 907/1799] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:39 2022] DELETE /identity/v3/projects/2de1b0eebe364e5ba03b28ed17aac2c1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:39.281324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.284481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.396570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:39.397232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.397514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.397786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.398524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:39.400533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929f576-c7f1-47bf-a797-0fd672da3435 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.461181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 893/1800] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:39 2022] POST /identity/v3/projects => generated 357 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:39.465150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.465724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.465724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.494118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23a4ef-0994-4e4d-bb1d-9090e1e9d516 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:39.508289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:39.514502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.570319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:39.690986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:39.695130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.695898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.696671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.701650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.713148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 894/1801] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:39.713148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.716025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.741016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:39.744036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b464d0-9eb4-4a6c-8daf-ca0df25df8fa service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.749911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109272b6-6993-4a5c-af24-583217b5f094 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:39.763300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:39.764389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.764800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.765198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.765982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:39.794265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 908/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:39.794807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.794888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.794888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.794888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.858646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 895/1803] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:39 2022] POST /identity/v3/users => generated 279 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:39.861955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.877207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:39.887626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:39.918684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ab6f-4928-456f-be60-233d7d305462 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:39.935527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 909/1804] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:39.947475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:39.951027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:39.953065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:39.954976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:39.957205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:39.959958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:39.966774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:39.988797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0614482-cb1f-4781-b87c-643932d53970 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.007369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.009566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.010243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.012326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.014223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 896/1805] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:40.014765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:40.015278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.017732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.022660 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.044344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108918-36cc-42f0-a83e-9a9a1decbcbb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.059188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 910/1806] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:40.061928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.063877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.066696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.067373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.069112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.070213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.076169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.101184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503519f-e92c-4306-b0fa-36dbe8d28662 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.138899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 897/1807] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:40.143572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.150594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.155384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.156024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.156519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.157781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.175668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.176601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a5ff6-e8ad-407b-a1d2-7449acfed100 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.187267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.188233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.189017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.190325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] PATH_INFO: `/v3/projects/a7e14dcc651e4ca89ae36ba87996b23d/users/a8cebef35be6496ba7735ca208925e8f/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.213012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e14dcc651e4ca89ae36ba87996b23d, user_id=a8cebef35be6496ba7735ca208925e8f, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.216425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707f76-6d07-46db-9b5e-9be7ec651ca7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.258427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 898/1808] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:40 2022] PUT /identity/v3/projects/a7e14dcc651e4ca89ae36ba87996b23d/users/a8cebef35be6496ba7735ca208925e8f/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:40.260364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.260831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.261470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.326528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 911/1809] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:40 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:40.330513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.345574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:40.354340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] MFA Rules not processed for user `a8cebef35be6496ba7735ca208925e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:40.358091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:40.543402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b384472-0874-4a11-b016-81b0ea872393 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.573183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.576151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.578991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.579527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.580343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.589875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.612386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 899/1810] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:40.613665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.661660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455878-43c1-4b5b-8b3e-500abb14f92e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.677220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 912/1811] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:40 2022] POST /identity/v3/users => generated 292 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:40.682234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.684560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.688043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:40.689219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.689636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.692854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.695337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.704339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.743775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f80e3-456b-412f-8308-c2bfa36b26e9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:40.767397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:40.768220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:40.768795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:40.791328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] PATH_INFO: `/v3/projects/742c8612be11476f92b38683ccd818bd` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:40.798287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] RBAC: Authorizing `identity:delete_project(project_id=742c8612be11476f92b38683ccd818bd)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:40.800776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:40.840900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 900/1812] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:40 2022] POST /identity/v3/projects => generated 355 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:40.848213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:40.967550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742c8612be11476f92b38683ccd818bd', 'request_id': 'req-e972eead-ae18-4135-afa1-031c5bb3370b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:40.988289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742c8612be11476f92b38683ccd818bd', 'request_id': 'req-e972eead-ae18-4135-afa1-031c5bb3370b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.014273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742c8612be11476f92b38683ccd818bd', 'request_id': 'req-e972eead-ae18-4135-afa1-031c5bb3370b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.021166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a730d-234f-476b-bb1b-77b37bdf59a6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.028931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:41.059483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] The token cache is being invalidate because project 742c8612be11476f92b38683ccd818bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:41.061834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.062460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.072179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 913/1813] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:40 2022] DELETE /identity/v3/projects/742c8612be11476f92b38683ccd818bd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:41.072179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.075193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:41.247576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.248235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.248515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.248788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.249490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.251296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.297046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972eead-ae18-4135-afa1-031c5bb3370b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.313368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.315605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.316631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.317898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.319033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.321451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.329648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 901/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:41.332681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.336123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:41.395934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 914/1815] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:41 2022] POST /identity/v3/users => generated 284 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:41.397560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb945-736d-44de-be0f-1a756b810192 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.402124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.406262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:41.431971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.433091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.433612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.434095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] PATH_INFO: `/v3/users/aadeb71e6fc34b9a9615d4211af86109` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.444125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadeb71e6fc34b9a9615d4211af86109)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.447952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.459646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913000d-c298-400b-8c51-a3c0f0de0caa admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.468965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.469848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.470237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.470752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.471827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.473984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.508516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 915/1816] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:41.509936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.526333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53de6-d19a-44ac-b3a6-73c6e7b82e2b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.538768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.540401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.540401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.541008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.541826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.544057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.595206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:41.601980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 916/1817] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:41.616499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.637687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bb5f3-eaeb-4d28-8c79-fe97019b9fc2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:41.662627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadeb71e6fc34b9a9615d4211af86109'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.669233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:41.670009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.670421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.670929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] PATH_INFO: `/v3/projects/fe991086995149b0a0ecaa1841245937/users/7087995d89e3409887ef84d073adc838/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:41.696579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadeb71e6fc34b9a9615d4211af86109'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.743576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe991086995149b0a0ecaa1841245937, user_id=7087995d89e3409887ef84d073adc838, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:41.750580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:41.758822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadeb71e6fc34b9a9615d4211af86109'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.761255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadeb71e6fc34b9a9615d4211af86109'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.788353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 917/1818] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:41 2022] PUT /identity/v3/projects/fe991086995149b0a0ecaa1841245937/users/7087995d89e3409887ef84d073adc838/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:41.789564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:41.792266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:41.793179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0a3c554a-c453-41d3-955c-fc5f13c37825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadeb71e6fc34b9a9615d4211af86109'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:41.858085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 902/1819] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:41 2022] DELETE /identity/v3/users/aadeb71e6fc34b9a9615d4211af86109 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:41.858085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64043-1967-426d-b63b-9c3d08f18ddb None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:41.865915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64043-1967-426d-b63b-9c3d08f18ddb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:41.867813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd64043-1967-426d-b63b-9c3d08f18ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.009860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1fd64043-1967-426d-b63b-9c3d08f18ddb None None] MFA Rules not processed for user `7087995d89e3409887ef84d073adc838`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:42.011156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1fd64043-1967-426d-b63b-9c3d08f18ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:42.012309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5b5b-eb79-4e3f-96f3-ff7e78439370 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:42.221669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:42.273431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae770-d397-4454-871b-698384701773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:42.274061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 903/1820] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:42.274779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae770-d397-4454-871b-698384701773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.275531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae770-d397-4454-871b-698384701773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.276004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ae770-d397-4454-871b-698384701773 admin admin] PATH_INFO: `/v3/projects/7e49b0dda20c4213a85d30c95b88e488/users/f6f526fcddaf4450bf5330b293334c88/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.276212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.276674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.277127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.283279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 904/1821] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:58:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:42.284422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.291700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:42.326786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae770-d397-4454-871b-698384701773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e49b0dda20c4213a85d30c95b88e488, user_id=f6f526fcddaf4450bf5330b293334c88, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:42.336540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ae770-d397-4454-871b-698384701773 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:42.348582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bcb18-f32b-4a83-b305-aede1460a420 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:42.371779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 918/1822] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:41 2022] PUT /identity/v3/projects/7e49b0dda20c4213a85d30c95b88e488/users/f6f526fcddaf4450bf5330b293334c88/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:42.376738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae770-d397-4454-871b-698384701773 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.383478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:42.384439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.394817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.395414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.396548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:42.405379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc298e-65a5-42ac-a35a-e5b7c999ff2b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:42.561066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 905/1823] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/projects => generated 387 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:42.562839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.570917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.571463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.619130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:42.620671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:42.622885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ae770-d397-4454-871b-698384701773 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:42.755977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:42.772551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:42.773307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.773665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.774577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.775378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:42.779221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:42.785052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 906/1824] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:42.786053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.815450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 919/1825] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:42.824879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.825525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1f5a6-c1b4-46d6-94fe-d23b38bbcc47 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:42.840608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:42.841334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.842451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.843139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.846982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:42.851209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab53e3-d763-43e0-9a4f-66e3987ddd40 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:42.861066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfaf49-20e2-4f6b-878b-f8894fa0a393 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:42.886670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:42.886670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.886670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.886670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] PATH_INFO: `/v3/projects/7e49b0dda20c4213a85d30c95b88e488/users/f6f526fcddaf4450bf5330b293334c88/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.888295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e49b0dda20c4213a85d30c95b88e488, user_id=f6f526fcddaf4450bf5330b293334c88, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:42.896277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad48603-6200-446c-b56a-cd15014591a1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:42.925600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 907/1826] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:42.928801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5a8d-332c-44b8-8ff5-633478166ea1 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.930136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5a8d-332c-44b8-8ff5-633478166ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.930511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5a8d-332c-44b8-8ff5-633478166ea1 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.933958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 908/1827] 10.222.0.52 () {54 vars in 961 bytes} [Tue Sep 13 19:58:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:42.937401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 920/1828] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:42 2022] PUT /identity/v3/projects/7e49b0dda20c4213a85d30c95b88e488/users/f6f526fcddaf4450bf5330b293334c88/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:42.951210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5a8d-332c-44b8-8ff5-633478166ea1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.952637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775e8f-5428-490c-96d5-55ebab3d368f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.953007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775e8f-5428-490c-96d5-55ebab3d368f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.953333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf775e8f-5428-490c-96d5-55ebab3d368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.038828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cf775e8f-5428-490c-96d5-55ebab3d368f None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:43.043187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cf775e8f-5428-490c-96d5-55ebab3d368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:43.244910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:43.305855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 921/1829] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:43.312516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.312516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.312926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.336433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5a8d-332c-44b8-8ff5-633478166ea1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.358640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.360191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.361071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.361858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.363854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.367646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.405139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] MFA Rules not processed for user `f6f526fcddaf4450bf5330b293334c88`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:43.407664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:43.419271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 909/1830] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 485 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:43.424264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.449273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc3790-f4c5-47b9-ba8f-462cbee482f4 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.456283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.457442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.458003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.458633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.481904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.487330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.526517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 910/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:43.528398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.554752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214e620-4ed7-406d-8e0d-28bc190dd9e6 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.568290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.569620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.570212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.570810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] PATH_INFO: `/v3/projects/f3c3a96a05b84012ace94b77ea4ee295/users/30ab7b6e69f3494ab3eddaeeda7e0513/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.623212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 922/1832] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:43.651343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.653181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:43.663952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c3a96a05b84012ace94b77ea4ee295, user_id=30ab7b6e69f3494ab3eddaeeda7e0513, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.677976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.745379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 911/1833] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:43 2022] PUT /identity/v3/projects/f3c3a96a05b84012ace94b77ea4ee295/users/30ab7b6e69f3494ab3eddaeeda7e0513/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:43.752369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.821648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22372f49-61bb-4c15-a5f6-531592bb99c7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.944178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.945038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.945461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.945855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:44.005107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ed98d-6002-4ae2-b945-fbc00f963e8a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:44.011706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:44.012862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:44.013313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:44.013809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:44.015498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:44.018960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:44.040607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 912/1834] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:44.044940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:44.060895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4147a-c2c9-4928-8883-a7e20d11c0a2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:44.063935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:44.069392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:44.070325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:44.070734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:44.071107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] PATH_INFO: `/v3/projects/f3c3a96a05b84012ace94b77ea4ee295/users/30ab7b6e69f3494ab3eddaeeda7e0513/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:44.074327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c3a96a05b84012ace94b77ea4ee295, user_id=30ab7b6e69f3494ab3eddaeeda7e0513, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:44.079496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c20641-6445-45f7-83cc-48a4269c3711 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:44.097089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:44.098624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:44.116109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 913/1835] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:44 2022] PUT /identity/v3/projects/f3c3a96a05b84012ace94b77ea4ee295/users/30ab7b6e69f3494ab3eddaeeda7e0513/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:44.128528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:44.129673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:44.130232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 923/1836] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:44.130490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:44.211281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] MFA Rules not processed for user `30ab7b6e69f3494ab3eddaeeda7e0513`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:44.216448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:44.271384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:44.319317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:44.332594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb361be-5baf-4f76-83e3-5a76b66d82fb service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:44.486462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:44.491898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:44.492695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:44.494642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:44.501037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:44.600077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 914/1837] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:44.608382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:44.634749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:44.813603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:44.816630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:44.861263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 924/1838] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:44.903681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:44.917090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:44.991869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48696b7-9d76-4348-b1e5-ffa482286f24 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:45.013831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:45.015419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:45.015716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:45.020112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:45.051547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedc196-79ef-4e30-be45-bd08e68c21bc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:45.069649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:45.072444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:45.074732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dae4a-6a73-4b02-aa7a-1bce60128808 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:45.074924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:45.074924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:45.075289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] PATH_INFO: `/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:45.078206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ad8d8a58594c2dbfb8c17c27ed2dce)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:45.082844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:45.101144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 925/1839] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:45.294416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ad8d8a58594c2dbfb8c17c27ed2dce', 'request_id': 'req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:45.336786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ad8d8a58594c2dbfb8c17c27ed2dce', 'request_id': 'req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:45.380329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ad8d8a58594c2dbfb8c17c27ed2dce', 'request_id': 'req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:45.425915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] The token cache is being invalidate because project 53ad8d8a58594c2dbfb8c17c27ed2dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:45.426963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:45.428589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6ea79c9f-e7f1-4ebc-8925-d28d0bdc742b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:45.433123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 915/1840] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:44 2022] DELETE /identity/v3/projects/53ad8d8a58594c2dbfb8c17c27ed2dce => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:45.439554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:45.440480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:45.440985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:45.504408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:45.511040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:45.855600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 926/1841] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:45.864103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:45.864699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:45.865223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:45.945205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:45.946845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:45.993560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 916/1842] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:45.995502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.042479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13899ebe-f17f-4f0e-8045-0eaa381d2dbc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.093078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.093803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.094086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.094406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.096244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:46.096588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.098543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5ba6e-ca0e-4cfc-834c-4d53495958f5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:46.157069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 927/1843] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:46.165557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.195652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9fac9-8884-4684-9c75-50c0391c3bd8 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.232231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.233455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.233455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.233632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.234499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.254027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2baefb-1850-492c-b22b-7c41ed5200ed admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:46.280270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73daf-768e-46e3-97ca-6441afdf2e92 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.280820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73daf-768e-46e3-97ca-6441afdf2e92 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.281243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a73daf-768e-46e3-97ca-6441afdf2e92 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.283866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 928/1844] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:58:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:46.324654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 917/1845] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:46 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:46.331827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73daf-768e-46e3-97ca-6441afdf2e92 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.371449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:46.527671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a73daf-768e-46e3-97ca-6441afdf2e92 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.545265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.545929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.546215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.546525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.547380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.554622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:46.615099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84395-f3d7-46f8-911a-84ca0d87f837 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.616567 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84395-f3d7-46f8-911a-84ca0d87f837 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.618179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c84395-f3d7-46f8-911a-84ca0d87f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.655204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 929/1846] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:46 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:46.657555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.682828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6b982-b6fb-4677-a568-94c261a6e5af admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.699045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.699967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.700427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.700861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.701695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.703675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:46.713256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f4c84395-f3d7-46f8-911a-84ca0d87f837 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:46.721052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f4c84395-f3d7-46f8-911a-84ca0d87f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:46.730591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 930/1847] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:46.736173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.755557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:46.756364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bb4ad-1628-45d9-b84b-ecf73bea7970 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.775274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.776192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.776617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.777142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.778203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.780323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:46.821867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 931/1848] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:46.824918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:46.878169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1f59-ac83-4842-80ac-f3941b69d1a4 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:46.901428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:46.904564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:46.906306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:46.909365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] PATH_INFO: `/v3/projects/cebe2ee393864913b048ff45dfa87d2d/users/1ca3511cf7e1471fa48d5b7f23a9c21c/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:46.985503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebe2ee393864913b048ff45dfa87d2d, user_id=1ca3511cf7e1471fa48d5b7f23a9c21c, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:46.999090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:47.106095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 918/1849] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:58:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:47.130539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 932/1850] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:46 2022] PUT /identity/v3/projects/cebe2ee393864913b048ff45dfa87d2d/users/1ca3511cf7e1471fa48d5b7f23a9c21c/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:47.141671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:47.142174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:47.143444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:47.281804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] MFA Rules not processed for user `1ca3511cf7e1471fa48d5b7f23a9c21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:47.287864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:47.320056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:47.324062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:47.531958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 919/1851] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:47.538973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:47.730280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22655aa2-9731-4e3e-9297-214fc2faec94 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:47.735841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:47.736495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:47.736778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:47.737052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:47.737719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:47.739832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dddb3a-c670-4385-86b9-31541639e015 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:47.753823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719786a-2fa2-44a6-b2fd-2af65891d372 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:47.786860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:47.787240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 920/1852] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:47 2022] POST /identity/v3/projects => generated 379 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:47.788774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c904d-f9af-4c13-8b44-8cd15b5483f0 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:47.789068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c904d-f9af-4c13-8b44-8cd15b5483f0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:47.789496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c904d-f9af-4c13-8b44-8cd15b5483f0 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:47.790236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:47.790817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:47.791966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] PATH_INFO: `/v3/users/7087995d89e3409887ef84d073adc838` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:47.794693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 921/1853] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:58:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:47.795676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c904d-f9af-4c13-8b44-8cd15b5483f0 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:47.799607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7087995d89e3409887ef84d073adc838)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:47.808040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:47.848976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:48.023253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c904d-f9af-4c13-8b44-8cd15b5483f0 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:48.031638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7087995d89e3409887ef84d073adc838'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:48.045578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:48.047130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.047624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.048146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.049672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:48.054174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4213-6d70-400f-a80a-fa432b8cdc4f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:48.060198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7087995d89e3409887ef84d073adc838'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:48.090105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:48.113569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7087995d89e3409887ef84d073adc838'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:48.117819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7087995d89e3409887ef84d073adc838'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:48.143737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7087995d89e3409887ef84d073adc838'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:48.153322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 922/1854] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:47 2022] POST /identity/v3/users => generated 290 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:48.156004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.156506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.157089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.164374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 933/1855] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:47 2022] DELETE /identity/v3/users/7087995d89e3409887ef84d073adc838 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:48.164374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.168330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.222989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] MFA Rules not processed for user `e7bec1fe5f26425eb9d3e98f7502a7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:48.225422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.381346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 923/1856] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:58:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:48.382348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.384313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.415787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8cec0-7b31-4f05-b1b3-e4ad5599eb65 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:48.435499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:48.436099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.436405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.436721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.440286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.477770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:48.480019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635e2e6-4e9a-452c-868f-173fb6d08c48 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:48.522888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 934/1857] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:48.524828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.525165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.525503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.563176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268220e2-6e2d-42c3-bd7a-a18b422c6afa None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:48.584388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:48.585656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.586160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.586877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.588338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:48.594857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:48.612851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:48.617298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.627483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 924/1858] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:48.629181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.634886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.649286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 935/1859] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:48.652920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.704199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d178e4-c9b2-41c3-9004-2538127370d1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:48.705433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0f42e-dada-4175-89b9-32d3ee056603 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:48.729857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:48.732153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:48.733893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.735273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.735950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:48.737157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.738113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:48.739730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:48.743307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:48.752210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:48.794096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:48.809413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:48.857484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 936/1860] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:48.866717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.886821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:48.947848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 925/1861] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:48.948958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:48.977927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:49.071614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eaab14-86ca-47d6-a7ef-dfe8fcb72d48 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:49.109077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:49.111557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:49.112102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:49.113210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] PATH_INFO: `/v3/users/f6ec10e0134641468b49558d9fb1cacc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:49.117657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ec10e0134641468b49558d9fb1cacc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:49.121727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:49.263100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3bbd7-e830-47c5-8764-5889c48a55ea admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:49.284415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:49.285625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:49.286268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:49.286876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:49.299986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:49.323373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:49.325271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:49.339778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 937/1862] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:49.341724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:49.364901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74001e1-da76-4480-9c23-c4b5e744794c service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:49.413868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:49.415657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:49.416324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:49.418886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] PATH_INFO: `/v3/projects/e0fd8216a7de45cc81c5d8185d80026e/users/32578e4bae9a47968aa78db07d96a129/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:49.429781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ec10e0134641468b49558d9fb1cacc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:49.444469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ec10e0134641468b49558d9fb1cacc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:49.460332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fd8216a7de45cc81c5d8185d80026e, user_id=32578e4bae9a47968aa78db07d96a129, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:49.466762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390449a-33a1-4a78-915b-b1265c241e19 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:49.483554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ec10e0134641468b49558d9fb1cacc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:49.501330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ec10e0134641468b49558d9fb1cacc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:49.519875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 938/1863] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:49 2022] PUT /identity/v3/projects/e0fd8216a7de45cc81c5d8185d80026e/users/32578e4bae9a47968aa78db07d96a129/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:49.528639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ec10e0134641468b49558d9fb1cacc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:49.543510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:49.543991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:49.544472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:49.560270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 926/1864] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:48 2022] DELETE /identity/v3/users/f6ec10e0134641468b49558d9fb1cacc => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:49.631940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] MFA Rules not processed for user `32578e4bae9a47968aa78db07d96a129`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:49.634473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:49.663526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:49.871897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 939/1865] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:49.872355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:49.875701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.140451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:50.148233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.175105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c63d28-c9d2-49c0-ad7c-d1abf93cc90b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:50.189427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:50.191003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.191941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.192826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.194295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:50.202526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f89e13-ed43-4dc6-81a0-29cc260be68b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:50.204256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:50.219829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:50.220665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.220957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.221236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] PATH_INFO: `/v3/users/6f32985d82194bb99d0d3411d212e4ad` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.222436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f32985d82194bb99d0d3411d212e4ad)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:50.224409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:50.261907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 927/1866] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:49 2022] POST /identity/v3/projects => generated 379 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:50.266134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:50.423318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f32985d82194bb99d0d3411d212e4ad'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:50.454399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f32985d82194bb99d0d3411d212e4ad'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:50.493189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18eb56-fbaf-48ee-9abc-3649cf1c3832 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:50.519989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f32985d82194bb99d0d3411d212e4ad'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:50.521451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f32985d82194bb99d0d3411d212e4ad'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:50.525547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:50.526435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.526947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.527371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.530919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:50.533197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:50.556781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-974c5e1e-29cc-46ed-b94f-e41967728886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f32985d82194bb99d0d3411d212e4ad'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:50.579658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 940/1867] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:50 2022] DELETE /identity/v3/users/6f32985d82194bb99d0d3411d212e4ad => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:50.582645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddaf26-8c2c-41e1-9ae4-8833867d7e5a None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.583323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddaf26-8c2c-41e1-9ae4-8833867d7e5a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.583976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddaf26-8c2c-41e1-9ae4-8833867d7e5a None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.588422 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 941/1868] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:58:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:50.590940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd47-f425-4ab1-ae62-5790d79a3940 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.591683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd47-f425-4ab1-ae62-5790d79a3940 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.592834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9cd47-f425-4ab1-ae62-5790d79a3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.619045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 928/1869] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:50 2022] POST /identity/v3/users => generated 296 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:50.625352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:50.629046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.636549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-aaa9cd47-f425-4ab1-ae62-5790d79a3940 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:50.638276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-aaa9cd47-f425-4ab1-ae62-5790d79a3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.653743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec148a-de6c-4eca-8b0a-9861733af0a7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:50.669266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:50.730964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 942/1870] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:50.732646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.733011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.733359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.772552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:50.773568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.774140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.774764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:50.775829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:50.782065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:50.783953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.789790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:50.819180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:50.863406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 929/1871] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:50.864586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:50.869208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:50.923690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 943/1872] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:58:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:50.924635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:50.948796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3767-69f2-4d99-a371-bb04df4bfce4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:50.960043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:50.965430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:50.965786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:50.966069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] PATH_INFO: `/v3/projects/b25e6721d28441f7b2190ba4f6c55468/users/0df49566f68e4db38c0f397590c2979e/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.055416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25e6721d28441f7b2190ba4f6c55468, user_id=0df49566f68e4db38c0f397590c2979e, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.058083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82efe0-9763-4139-9ebc-90e6103e95bf admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.060429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.066164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.067090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.067469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.067839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.091898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.095880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.099047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 944/1873] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:50 2022] PUT /identity/v3/projects/b25e6721d28441f7b2190ba4f6c55468/users/0df49566f68e4db38c0f397590c2979e/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:51.100012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.108818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.128858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 930/1874] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:51.129964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.131999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.169885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553035f-22c6-4731-bff5-bf03a072e4cf service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.246463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdc9e-2642-45ce-b076-db15923a040d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.259341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.260033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.260335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.260586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] PATH_INFO: `/v3/projects/7cef6e663f8849e4bbff2bc7a77f11e6` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.262045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cef6e663f8849e4bbff2bc7a77f11e6)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.264176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.294679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.295968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.296590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.297103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.311512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.316576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.328210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cef6e663f8849e4bbff2bc7a77f11e6', 'request_id': 'req-e0934e00-35f3-467a-befe-f2d9420c4d9c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.336298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cef6e663f8849e4bbff2bc7a77f11e6', 'request_id': 'req-e0934e00-35f3-467a-befe-f2d9420c4d9c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.336666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 931/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:51.338125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.347975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cef6e663f8849e4bbff2bc7a77f11e6', 'request_id': 'req-e0934e00-35f3-467a-befe-f2d9420c4d9c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.353432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267461d5-f36a-45d5-8c6d-66cc464c1899 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.383040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] The token cache is being invalidate because project 7cef6e663f8849e4bbff2bc7a77f11e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:51.383784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.384690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.389517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 945/1876] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:51 2022] DELETE /identity/v3/projects/7cef6e663f8849e4bbff2bc7a77f11e6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:51.394291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.396108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.420958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:51.457930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.458846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.459942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:51.460734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.461386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.462775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.466488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.491626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 932/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:51.492965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.497064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.590118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23a3c-13e1-40e6-84b5-e2bad447b8bb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.623155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.626405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.627675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.628495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] PATH_INFO: `/v3/projects/b25e6721d28441f7b2190ba4f6c55468/users/0df49566f68e4db38c0f397590c2979e/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.643043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25e6721d28441f7b2190ba4f6c55468, user_id=0df49566f68e4db38c0f397590c2979e, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.649133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59dc29-e57f-47d1-b86e-8469ed93098e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.675650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0934e00-35f3-467a-befe-f2d9420c4d9c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.698288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 933/1878] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:51 2022] PUT /identity/v3/projects/b25e6721d28441f7b2190ba4f6c55468/users/0df49566f68e4db38c0f397590c2979e/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:51.713503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.714087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.714087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.784334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.785014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.785293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.785567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] PATH_INFO: `/v3/users/5874d4f28dc140fbb8fe22606318ac5d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.786847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5874d4f28dc140fbb8fe22606318ac5d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.789055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.807392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] MFA Rules not processed for user `0df49566f68e4db38c0f397590c2979e`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:51.808732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.867934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5874d4f28dc140fbb8fe22606318ac5d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.905083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5874d4f28dc140fbb8fe22606318ac5d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.944449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5874d4f28dc140fbb8fe22606318ac5d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.949232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5874d4f28dc140fbb8fe22606318ac5d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:51.949982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 934/1879] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:51.951810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.955653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.002241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5874d4f28dc140fbb8fe22606318ac5d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.036703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 946/1880] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:51 2022] DELETE /identity/v3/users/5874d4f28dc140fbb8fe22606318ac5d => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:52.037838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:52.043090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abf913-ce0d-442d-b5a7-5349e66e07ea None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.047520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.196792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:52.204899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2504ce2-61a1-4fb5-a7e1-877779b7f28b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.216418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.217814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.218550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.218830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] PATH_INFO: `/v3/projects/fe991086995149b0a0ecaa1841245937` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.220448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe991086995149b0a0ecaa1841245937)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.224580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.226967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.227635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.228083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.228360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.230732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.308153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:52.310912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe991086995149b0a0ecaa1841245937', 'request_id': 'req-8ecda06a-8a98-45f7-9675-833bfe1e9a04'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.322067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe991086995149b0a0ecaa1841245937', 'request_id': 'req-8ecda06a-8a98-45f7-9675-833bfe1e9a04'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.333205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe991086995149b0a0ecaa1841245937', 'request_id': 'req-8ecda06a-8a98-45f7-9675-833bfe1e9a04'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.362678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] The token cache is being invalidate because project fe991086995149b0a0ecaa1841245937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:52.362964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.363951 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.369652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 935/1881] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:51 2022] DELETE /identity/v3/projects/fe991086995149b0a0ecaa1841245937 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:52.376204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:52.378174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.378463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.381482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.393821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 947/1882] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:52.475116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda06a-8a98-45f7-9675-833bfe1e9a04 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.486778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:52.487680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.487680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.488244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.488401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.489439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] PATH_INFO: `/v3/users/30ab7b6e69f3494ab3eddaeeda7e0513` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.491181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ab7b6e69f3494ab3eddaeeda7e0513)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.493182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.514430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9191be3-7d9e-41be-af86-546e45693b9a service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.611116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ab7b6e69f3494ab3eddaeeda7e0513'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.624255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ab7b6e69f3494ab3eddaeeda7e0513'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.637043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.637982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.638395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.638936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.643686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.649507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ab7b6e69f3494ab3eddaeeda7e0513'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.650832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ab7b6e69f3494ab3eddaeeda7e0513'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.661184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ab7b6e69f3494ab3eddaeeda7e0513'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.668760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 936/1883] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:52 2022] DELETE /identity/v3/users/30ab7b6e69f3494ab3eddaeeda7e0513 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:52.669557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:52.671235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.759230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.765132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.774175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:52.798143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 948/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:52.799083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:52.802641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2ce0d-b709-4c55-9df6-113792c4b263 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.804838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:52.825638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.826602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.826933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.827197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] PATH_INFO: `/v3/projects/90f39579107b4134acedd96b3dd97ecb` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.833329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f39579107b4134acedd96b3dd97ecb)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.836305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.857224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bc417-1431-4a06-a518-47fb2d632fdb service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:52.872715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:52.873486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:52.874225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:52.874225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] PATH_INFO: `/v3/projects/e668623112ac436ea9dc0da7d83c68c1` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:52.881308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e668623112ac436ea9dc0da7d83c68c1)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:52.885214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:52.968510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f39579107b4134acedd96b3dd97ecb', 'request_id': 'req-e55709ae-098a-4a3b-a040-6910147ecba4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:52.979706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f39579107b4134acedd96b3dd97ecb', 'request_id': 'req-e55709ae-098a-4a3b-a040-6910147ecba4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.009732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:53.014542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f39579107b4134acedd96b3dd97ecb', 'request_id': 'req-e55709ae-098a-4a3b-a040-6910147ecba4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.021905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e668623112ac436ea9dc0da7d83c68c1', 'request_id': 'req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.036141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e668623112ac436ea9dc0da7d83c68c1', 'request_id': 'req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.048675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e668623112ac436ea9dc0da7d83c68c1', 'request_id': 'req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.058747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] The token cache is being invalidate because project 90f39579107b4134acedd96b3dd97ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:53.061329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.063919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e55709ae-098a-4a3b-a040-6910147ecba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.068196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] The token cache is being invalidate because project e668623112ac436ea9dc0da7d83c68c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:53.068390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.069367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 937/1885] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:52 2022] DELETE /identity/v3/projects/90f39579107b4134acedd96b3dd97ecb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:53.070238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c4429ea8-a2c4-48ec-828e-97c3f7126d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:53.080217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.081625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 949/1886] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:52 2022] DELETE /identity/v3/projects/e668623112ac436ea9dc0da7d83c68c1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:53.082359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.083149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.093112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be361c-1a7a-4bb5-8046-2ef483478217 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.094940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be361c-1a7a-4bb5-8046-2ef483478217 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.095641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be361c-1a7a-4bb5-8046-2ef483478217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.146020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-67be361c-1a7a-4bb5-8046-2ef483478217 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:53.150862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-67be361c-1a7a-4bb5-8046-2ef483478217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.182513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:53.183869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.309743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 950/1887] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:53.312048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6715bd-75bf-4791-a033-44631cd73ac0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.312421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6715bd-75bf-4791-a033-44631cd73ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.312761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6715bd-75bf-4791-a033-44631cd73ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.341685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2c6715bd-75bf-4791-a033-44631cd73ac0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:53.343146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2c6715bd-75bf-4791-a033-44631cd73ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.355207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 951/1889] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:53.355508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 938/1889] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:53.356549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.356810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.357087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.359620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.390520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99394131-bc58-4560-800e-36d6d8fbef44 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.400213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:53.404189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.415778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 952/1890] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:53.425991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.438840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.439648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.440037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.442967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.443849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:53.444267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.446038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.446516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c16497-87d5-4633-aa6f-c4bdb358747e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.465119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.465896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.466196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.466560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.468241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:53.468641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.473415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.490220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 939/1891] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:53.493434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.510462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 953/1892] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:53.516325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.525938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93579f6-7a38-44cb-9732-6a01b6a0531e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.540733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.554923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.555994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.556494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.558261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.561404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.576040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a60b2-a7e5-4b43-b643-a58ef19cc3ee admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.599429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.600157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.600440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.600715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.606025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.609051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.621813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 940/1893] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:53.621813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.625770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.682171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 954/1894] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:53.687215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.786327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:53.812446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdbb72-007b-46ae-ab34-197c789ef2a9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.850917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c06cdc-2a0a-471e-a841-15319a7c820a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.863648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.864511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.864795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.865110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.865824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.868034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.902172 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 955/1895] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:53.903818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:53.917083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25d4f-d57e-4630-8c3b-35cf8c25bec5 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:53.920463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.922675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.924356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.925138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:53.925548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] PATH_INFO: `/v3/projects/f3c3a96a05b84012ace94b77ea4ee295` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.925871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.926148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.927386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c3a96a05b84012ace94b77ea4ee295)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.929831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.931569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.932410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:53.934868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:53.971777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=3,keystone:INSERT=8 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:53.990029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 956/1896] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/users => generated 287 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:53.990029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.015895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4a38f-d2a1-4acc-a879-b614d722b783 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.029154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c3a96a05b84012ace94b77ea4ee295', 'request_id': 'req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:54.039466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c3a96a05b84012ace94b77ea4ee295', 'request_id': 'req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:54.049551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c3a96a05b84012ace94b77ea4ee295', 'request_id': 'req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:54.073387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] The token cache is being invalidate because project f3c3a96a05b84012ace94b77ea4ee295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:54.073843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:54.074714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:54.077172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 941/1897] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:53 2022] DELETE /identity/v3/projects/f3c3a96a05b84012ace94b77ea4ee295 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:54.082318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.084688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.108921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.111314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.112632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.113238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.117477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.122309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c148a36-313f-47ec-a6b7-1c3189f64906 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.138136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 957/1898] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:54.144938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.145247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.147088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.172933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d5bc6-2b59-45b9-9a75-5474d3beba52 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.175214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:54.176675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.178326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.179016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.179303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.179586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.180257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.181898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ada28d-ed1d-483e-8009-a5a248130f19 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.186181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 958/1899] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:54.187135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.188522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.195027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 942/1900] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:54.196780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.197071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.197349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.207390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ad8eb-037c-45bf-bdfe-fab619527900 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.214478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.215201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.215564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.215835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.216747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.218939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.224736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:54.226785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.232993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 959/1901] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:54.234336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.236505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 943/1902] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:54.237830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.249371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3928e-d0da-4cea-9514-ad822846e836 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.249712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191347c-57a8-495b-8642-0b88a1b3bce1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.254928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.255771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.256105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.256543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.256732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.257260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.257609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.257840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] PATH_INFO: `/v3/projects/64a19d6ad79f4b6397cbd7e5bad1b75b/users/bff59be9006440bfb0efb34c45e138c4/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.258447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.260387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.273285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 960/1903] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:54.274124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.283743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a19d6ad79f4b6397cbd7e5bad1b75b, user_id=bff59be9006440bfb0efb34c45e138c4, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.285510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7790a6-9194-47f0-9a7e-a1171e4d46dc admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.287780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.291027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.291867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.292168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.292273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:54.292376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.292876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:54.293186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.295301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dea73-d81b-4927-9721-6a5c17dd4d3d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.312247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 944/1904] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:54 2022] PUT /identity/v3/projects/64a19d6ad79f4b6397cbd7e5bad1b75b/users/bff59be9006440bfb0efb34c45e138c4/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:54.314038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 961/1905] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:54.314863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.326083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.326083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.326646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.364853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] MFA Rules not processed for user `bff59be9006440bfb0efb34c45e138c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:54.366225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.478107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a60a6-a3c2-4f11-86ba-ea51dd6a02f2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.492361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.497699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.497699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.498420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] PATH_INFO: `/v3/projects/d56091e6d4da453fbeea125f6659f11d/users/0f08f07de04f4abb9f51d178b0f95bb7/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.516799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 962/1906] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:54.522311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.546617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f339e-9a51-4388-84d6-b9a12b50c02e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.550064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56091e6d4da453fbeea125f6659f11d, user_id=0f08f07de04f4abb9f51d178b0f95bb7, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.553262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.553483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.554191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.554514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.554788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.555489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.558007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93fb94-f220-4131-ae34-7b6ae0135f69 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.603004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 945/1907] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:54 2022] PUT /identity/v3/projects/d56091e6d4da453fbeea125f6659f11d/users/0f08f07de04f4abb9f51d178b0f95bb7/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:54.606688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.612870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 963/1908] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:54.616843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.617115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.617380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.736976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] MFA Rules not processed for user `0f08f07de04f4abb9f51d178b0f95bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:54.746519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.775851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:54.839295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c1bcf-79be-41ac-ad93-15e884a9ab6a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.980230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 964/1909] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:54.982620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.003016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.003673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.006049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.006481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.007999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.011667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.097183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 946/1910] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/projects => generated 377 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:55.098315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.114937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:55.194866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9826f-ff23-43f1-9e16-16ac2b51180e None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.340445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99294f2-bf6b-451b-80d1-b8d853096853 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.347990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.350109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.350673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.351173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.353505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.356173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.405850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 947/1911] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:55 2022] POST /identity/v3/projects => generated 371 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:55.408153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.447538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.450945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.451430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.451825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.453732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.458117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.583994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b9878-3ce4-47ae-8d9a-e748bc5871fa admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.596761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.596980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 965/1912] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:54 2022] POST /identity/v3/users => generated 286 bytes in 616 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:55.597714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.600858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.601141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.601776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.602070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.604581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.619913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad34d2d-12cf-4288-b5a5-db7c64f7616d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.633327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.634697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.635173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.635750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.639219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.647580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.650426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 948/1913] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:55 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:55.653709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.683455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1772f6-752a-4f29-be33-33990a87966f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.696340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 966/1914] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:55 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:55.697205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.700258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.702156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.702461 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.702746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.703766 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.705891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.716624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03041cc-4f19-4259-aaa8-ded9f3e9c469 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.723157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.724025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.724410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.725086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.725968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.728039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.739400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 949/1915] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:55.744221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.757089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46835a-bc2d-447c-88b1-69435d196f2f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.758550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 967/1916] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:55.759622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.771755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.772464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.772812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.773164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.773857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.775518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.790376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 950/1917] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:55.791249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.800698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28970-aba2-4c29-818e-d23fb1aa2bb6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.809450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03df88b-7b43-4a41-a28e-1fdfbbd60c37 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.818155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.818155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.818155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.818155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.819224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.819563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.824287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.825131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.825533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.825901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] PATH_INFO: `/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d/users/795ae5ad90a5455cb85dbccd7e184873/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.839177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 968/1918] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:55.840120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.848555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e9ec11146d4be98f7d2ae02b01c97d, user_id=795ae5ad90a5455cb85dbccd7e184873, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.853141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.856863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ad309-612b-405b-bdcb-89b4d82ead55 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:55.872819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 951/1919] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:55 2022] PUT /identity/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d/users/795ae5ad90a5455cb85dbccd7e184873/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:55.873836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.895584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:55.896398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:55.896796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:55.897529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] PATH_INFO: `/v3/projects/c2644b24b54243f2a5a53e1b25b74616/users/61870972789541e1a37ac0fb40624e70/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:55.901421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:55.923770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2644b24b54243f2a5a53e1b25b74616, user_id=61870972789541e1a37ac0fb40624e70, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:55.927767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:55.939217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:55.948381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 969/1920] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:55 2022] PUT /identity/v3/projects/c2644b24b54243f2a5a53e1b25b74616/users/61870972789541e1a37ac0fb40624e70/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:55.949415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:55.990630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6e4d8-ea78-4d47-9df0-acca642059be admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.061908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2bb01-8b7b-49de-9d73-aeae264292e4 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.068174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.070204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.070468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.070743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.071809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.073871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.086181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 970/1921] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:56.087165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.093070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.094315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.095102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.095449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] PATH_INFO: `/v3/projects/5ff3e012b3e34c3e9a94fe36153d24ae/users/7ab86646c648497194337640a9cda493/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.103762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7988738-242c-46ef-b692-b0e57e781805 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.110881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.113578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.113935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.114593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.116838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.118625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.126472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff3e012b3e34c3e9a94fe36153d24ae, user_id=7ab86646c648497194337640a9cda493, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.129921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.132237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 971/1922] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:56.133527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.148319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be01b9-894c-40ed-9fb3-420bfdb8449f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.152611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 952/1923] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:55 2022] PUT /identity/v3/projects/5ff3e012b3e34c3e9a94fe36153d24ae/users/7ab86646c648497194337640a9cda493/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:56.156836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.238174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.239499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.239700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.240552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] PATH_INFO: `/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d/users/795ae5ad90a5455cb85dbccd7e184873/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.242763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e9ec11146d4be98f7d2ae02b01c97d, user_id=795ae5ad90a5455cb85dbccd7e184873, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.245756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c513b-2fde-4025-bb09-2fb4fa96eb65 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.250450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99674695-852a-48b1-9abe-50e66ec91a22 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.257161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.257870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.258255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.258574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] PATH_INFO: `/v3/projects/c2644b24b54243f2a5a53e1b25b74616/users/61870972789541e1a37ac0fb40624e70/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.260757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2644b24b54243f2a5a53e1b25b74616, user_id=61870972789541e1a37ac0fb40624e70, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.261520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 972/1924] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:56 2022] PUT /identity/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d/users/795ae5ad90a5455cb85dbccd7e184873/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:56.263036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442fe4d-cbcf-4c35-9b5d-e2c89a2f79c2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.263822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442fe4d-cbcf-4c35-9b5d-e2c89a2f79c2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.263822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442fe4d-cbcf-4c35-9b5d-e2c89a2f79c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.274194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465f6c2-3876-4100-bf6e-6bf31a0c2476 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.290769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 953/1925] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:56 2022] PUT /identity/v3/projects/c2644b24b54243f2a5a53e1b25b74616/users/61870972789541e1a37ac0fb40624e70/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:56.294551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.297675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.299993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.304639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0442fe4d-cbcf-4c35-9b5d-e2c89a2f79c2 None None] MFA Rules not processed for user `7ab86646c648497194337640a9cda493`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:56.306638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0442fe4d-cbcf-4c35-9b5d-e2c89a2f79c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:56.355006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] MFA Rules not processed for user `795ae5ad90a5455cb85dbccd7e184873`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:56.357276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:56.391970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:56.417196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 973/1926] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:56.417838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.418076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.418386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.446392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:56.454680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] MFA Rules not processed for user `61870972789541e1a37ac0fb40624e70`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:56.456189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:56.482636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 954/1927] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:56.483824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.485472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:56.587341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 974/1928] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:56.588251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd6709-ed98-46d3-abb9-2d43ec861aa3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.588365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.590026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:56.595282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.596010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.596294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.596573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.609949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54114b-3bb4-4489-8905-e8b1ef895300 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.614676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.615344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.615628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.616056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.624564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.626098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.650424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 955/1929] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:56.693661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.785472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.787110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.813150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 975/1930] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:56.814555 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:56.828788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644217b-e6a6-4e59-b9c1-7aa6a142d7a3 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.836479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.837647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.838189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.838710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.852409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.854195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c373bc-b77b-48ca-a525-faca62b21e1b service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.866999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9a52-eea0-4773-9fe8-1cd35e4fedd0 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:56.872150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 976/1931] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:56.887390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:56.888104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:56.888386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:56.888660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] PATH_INFO: `/v3/users/7ab86646c648497194337640a9cda493` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:56.894550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab86646c648497194337640a9cda493)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:56.896611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:56.993655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab86646c648497194337640a9cda493'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.008926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab86646c648497194337640a9cda493'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.034077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab86646c648497194337640a9cda493'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.035053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab86646c648497194337640a9cda493'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.053280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-81d8669b-777c-4af6-a1b6-a939efe217b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab86646c648497194337640a9cda493'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.065670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 956/1932] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:56 2022] DELETE /identity/v3/users/7ab86646c648497194337640a9cda493 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:57.067627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.067926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.068209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.104420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973bda-18fc-4cb0-8889-3189b5fc005a None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.104975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973bda-18fc-4cb0-8889-3189b5fc005a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.106303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973bda-18fc-4cb0-8889-3189b5fc005a None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.113656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 957/1933] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 19:58:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:58:57.117218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:57.121139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.128105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.128467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.130001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.160618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:57.161946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.169230 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:57.318162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 977/1934] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:57.329118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:57.330831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.345159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:57.353918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 958/1935] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 19:58:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:57.372081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:57.373933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.398626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a49cb9-bfee-4d07-9804-2ffaa22eea21 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:57.412546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:57.413194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.413523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.413798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.445169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:57.450132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:57.483508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 959/1936] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:57.518800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afca12-acce-4eaa-83ca-3968228c7fe1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:57.529571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:57.530671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.531719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.533623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.574338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:57.577497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:57.579889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:57.582676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.630768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 978/1937] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:57.728597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea2e6-ce89-4944-b3ad-536be4702a0a service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:57.743307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:57.744068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.745873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.749193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.754511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:57.756030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.761112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:57.777239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672911c-61b6-4a1c-8320-3cd776335c8f service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:57.783473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:57.784560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.785032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.785406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] PATH_INFO: `/v3/users/1ca3511cf7e1471fa48d5b7f23a9c21c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.787139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca3511cf7e1471fa48d5b7f23a9c21c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:57.791302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:57.882437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca3511cf7e1471fa48d5b7f23a9c21c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.885593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:57.887304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580737a8-bf4f-4444-8276-8fe3276bcd02 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:57.902675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca3511cf7e1471fa48d5b7f23a9c21c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.925535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca3511cf7e1471fa48d5b7f23a9c21c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.926309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca3511cf7e1471fa48d5b7f23a9c21c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.931619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 960/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:57.937432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca3511cf7e1471fa48d5b7f23a9c21c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:57.952774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 979/1939] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:57 2022] DELETE /identity/v3/users/1ca3511cf7e1471fa48d5b7f23a9c21c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:57.954768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.955122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.955464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.005261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:58.008935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.108347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:58.182551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 961/1940] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:58.198792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.199894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.235881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.237304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.249501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7eaa1-64bf-495b-9991-24c2c95aff0f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.265146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:58.266607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.266958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.267366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] PATH_INFO: `/v3/projects/5ff3e012b3e34c3e9a94fe36153d24ae` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.268703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff3e012b3e34c3e9a94fe36153d24ae)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:58.270990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:58.271412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b8176-34f4-4a8a-821a-9686755c5fbd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.275808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:58.276537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.276876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.277209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.291067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:58.292939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a4dd3-d28b-4633-a38c-82b369d2e20f service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:58.308631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 980/1941] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:58.331454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff3e012b3e34c3e9a94fe36153d24ae', 'request_id': 'req-91e6ad63-de44-42ef-b651-e0f2ea810f2f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.344342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff3e012b3e34c3e9a94fe36153d24ae', 'request_id': 'req-91e6ad63-de44-42ef-b651-e0f2ea810f2f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.353253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff3e012b3e34c3e9a94fe36153d24ae', 'request_id': 'req-91e6ad63-de44-42ef-b651-e0f2ea810f2f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.375944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] The token cache is being invalidate because project 5ff3e012b3e34c3e9a94fe36153d24ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:58.376312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.377833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-91e6ad63-de44-42ef-b651-e0f2ea810f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.382431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 962/1942] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:58 2022] DELETE /identity/v3/projects/5ff3e012b3e34c3e9a94fe36153d24ae => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:58.389702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.389906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.390602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.427388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:58.428844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.454748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:58.514210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 981/1943] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:58.523777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.524391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.524724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.553926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:58.555542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.581915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 963/1944] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:58.599321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.615503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be564dc1-e2a5-435c-bcb3-98230f342161 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.624388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:58.626044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.627400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.627841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.629676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:58.630070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:58.632517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:58.649413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 982/1945] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:58.653895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.663868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3486fbe-d2b9-4641-b79b-5a7f21185594 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.669396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:58.670026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.670321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.670637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.671503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:58.674438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:58.696312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.698840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:58.723097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c33ec-5e81-4c59-87a8-7fdc98c0a1bf admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.735691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:58.736760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:58.737209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:58.737656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] PATH_INFO: `/v3/projects/cebe2ee393864913b048ff45dfa87d2d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:58.741317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebe2ee393864913b048ff45dfa87d2d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:58.741437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 964/1946] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:58 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:58.743141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.744493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:58.865433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebe2ee393864913b048ff45dfa87d2d', 'request_id': 'req-9e0a358f-511f-449d-856d-1cdbddbdce82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.892215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebe2ee393864913b048ff45dfa87d2d', 'request_id': 'req-9e0a358f-511f-449d-856d-1cdbddbdce82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.921044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebe2ee393864913b048ff45dfa87d2d', 'request_id': 'req-9e0a358f-511f-449d-856d-1cdbddbdce82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.938237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed5b43-026e-45ba-89b0-ed28f5a68a6f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:58.960291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] The token cache is being invalidate because project cebe2ee393864913b048ff45dfa87d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:58.961380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.961823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:58.963685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 983/1947] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:58 2022] DELETE /identity/v3/projects/cebe2ee393864913b048ff45dfa87d2d => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:58.968350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:58.972988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.024736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:59.050892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.051796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.052144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.052580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.054770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.057354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.089614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 965/1948] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:58 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:59.091122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.093781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.108206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a358f-511f-449d-856d-1cdbddbdce82 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.109444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18493-d4d5-4470-a915-d8735afce927 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.114998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.116002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.116284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.116452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.116561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] PATH_INFO: `/v3/users/32578e4bae9a47968aa78db07d96a129` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.117225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.117513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.117728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32578e4bae9a47968aa78db07d96a129)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.117835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.118473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.119783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.120395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.140474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 966/1949] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:59.144723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.179183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:59.190434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db5b8-cc88-44c6-b6eb-07c9e6125a88 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.192300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32578e4bae9a47968aa78db07d96a129'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:59.205397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.206171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.206354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32578e4bae9a47968aa78db07d96a129'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:59.206476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.206824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.207597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.211680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.226828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32578e4bae9a47968aa78db07d96a129'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:59.228388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32578e4bae9a47968aa78db07d96a129'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:59.243861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f40c8afc-4945-4ed6-bc2b-092330e39f7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32578e4bae9a47968aa78db07d96a129'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:59.245780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 967/1950] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:59.250424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.254829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.257972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 984/1951] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:58 2022] DELETE /identity/v3/users/32578e4bae9a47968aa78db07d96a129 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:59.279910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1adf-9fef-495c-9b9e-41455d9f07d1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.389511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.391409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.391704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.391966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] PATH_INFO: `/v3/projects/da9b406b6ac14778ac583ef4c204f588/users/4d14eab58e924a5db5eca2f2a43c4a36/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.429768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9b406b6ac14778ac583ef4c204f588, user_id=4d14eab58e924a5db5eca2f2a43c4a36, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.436620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.475142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 968/1952] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:59 2022] PUT /identity/v3/projects/da9b406b6ac14778ac583ef4c204f588/users/4d14eab58e924a5db5eca2f2a43c4a36/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:59.485605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.485890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.486473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.536384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] MFA Rules not processed for user `4d14eab58e924a5db5eca2f2a43c4a36`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:59.537916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.645211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 985/1953] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:58:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:59.665258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.668134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.746441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f765-fe40-486b-8a8c-bdc43e222cda admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.751089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.751773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.752325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.752325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.763029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.764779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.795304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 969/1954] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:59.844961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.846552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:59.944660 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8d8ed-26e7-4946-a3de-82a5422bd3e5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.950754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.950754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.951045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.954633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] PATH_INFO: `/v3/projects/e0fd8216a7de45cc81c5d8185d80026e` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.956199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fd8216a7de45cc81c5d8185d80026e)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.958689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:00.023416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:00.029918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fd8216a7de45cc81c5d8185d80026e', 'request_id': 'req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.038111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fd8216a7de45cc81c5d8185d80026e', 'request_id': 'req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.045246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fd8216a7de45cc81c5d8185d80026e', 'request_id': 'req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.062657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] The token cache is being invalidate because project e0fd8216a7de45cc81c5d8185d80026e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:00.063127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.063965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.070171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 986/1955] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:59 2022] DELETE /identity/v3/projects/e0fd8216a7de45cc81c5d8185d80026e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:00.071535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:00.086097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984b684-1722-4afa-b867-ff58a953f910 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:00.133255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:00.173180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:00.176031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.176031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.176031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] PATH_INFO: `/v3/users/0df49566f68e4db38c0f397590c2979e` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.177294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df49566f68e4db38c0f397590c2979e)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:00.179663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:00.240583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df49566f68e4db38c0f397590c2979e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.249059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df49566f68e4db38c0f397590c2979e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.261606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df49566f68e4db38c0f397590c2979e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.262220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df49566f68e4db38c0f397590c2979e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.269593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-2474a39d-a739-4bba-a238-904c1726205f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df49566f68e4db38c0f397590c2979e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.275546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 970/1956] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:00 2022] DELETE /identity/v3/users/0df49566f68e4db38c0f397590c2979e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:00.603882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:00.605695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:00.620350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed0462-cb5f-4dbf-a3fd-c241377b7246 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:00.691107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:00.691710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.691880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.692152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] PATH_INFO: `/v3/projects/b25e6721d28441f7b2190ba4f6c55468` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.693321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25e6721d28441f7b2190ba4f6c55468)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:00.695281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:00.748597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25e6721d28441f7b2190ba4f6c55468', 'request_id': 'req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.756097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25e6721d28441f7b2190ba4f6c55468', 'request_id': 'req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.763241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25e6721d28441f7b2190ba4f6c55468', 'request_id': 'req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.777991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] The token cache is being invalidate because project b25e6721d28441f7b2190ba4f6c55468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:00.778239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.778978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a05e2b33-5efa-4bec-9a80-4903ccdf1cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:00.780598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 987/1957] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:00 2022] DELETE /identity/v3/projects/b25e6721d28441f7b2190ba4f6c55468 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:00.787329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738af78-7095-41fd-a139-0467dab45fef None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.787627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738af78-7095-41fd-a139-0467dab45fef None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.787910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738af78-7095-41fd-a139-0467dab45fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.809868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2738af78-7095-41fd-a139-0467dab45fef None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:00.811148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2738af78-7095-41fd-a139-0467dab45fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:00.877004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 971/1958] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:00.889777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.890257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.890720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.919380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:00.920624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:00.929654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 988/1959] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:00.934505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738af78-7095-41fd-a139-0467dab45fef None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:00.944146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738af78-7095-41fd-a139-0467dab45fef None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:00.949150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:00.949790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.950043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.950315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.951258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:00.951353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:00.953006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:00.969657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 972/1960] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:00.970870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:00.981154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a39ea-1a9e-4069-9f09-b94719ce07b1 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:00.988003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:00.988703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:00.988986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:00.989309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:00.990057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:00.992309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.024423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 989/1961] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:00 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:01.028736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.039719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb831a-c501-4ae3-94e1-ff3c94681957 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.080555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:01.099316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.099316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.099316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.099316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.102926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.102926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.126789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 973/1962] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:01 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:01.131386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.142018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b590d0-ff30-4697-a348-f5637a67a807 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.147265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.147889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.148171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.148444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.149094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.150757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.161351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 990/1963] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:01.166027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.177063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8627-cbae-4a9d-87b6-0817e81d1d51 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.183250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.183975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.184299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.184720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.185847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.188328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10481552-4b90-4065-89cf-128c3241d256 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.204761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 974/1964] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:01.209148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.219917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bed65a-ff66-4584-a82c-4c2a3e153d55 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.225143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.225813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.226099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.226393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] PATH_INFO: `/v3/projects/390f81ebb576487d90378d341336ec3f/users/c1c26ee1c88940c5bf3d503b7aaf91d3/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.241802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] RBAC: Authorizing `identity:create_grant(project_id=390f81ebb576487d90378d341336ec3f, user_id=c1c26ee1c88940c5bf3d503b7aaf91d3, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.244576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.254661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:01.260776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 991/1965] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:01 2022] PUT /identity/v3/projects/390f81ebb576487d90378d341336ec3f/users/c1c26ee1c88940c5bf3d503b7aaf91d3/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:01.274481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.274774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.275083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.304096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] MFA Rules not processed for user `c1c26ee1c88940c5bf3d503b7aaf91d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:01.305356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:01.376636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 975/1966] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:01.381261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.443467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ba6e-6eec-45a3-823f-e9db1378127c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.448325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.448968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.449248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.449522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.450173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.452153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.481322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 992/1967] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:01 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:01.487536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.500526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d55ff1-c54a-40ee-9f7f-8ae6956434e4 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.570553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.572240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.572240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.572240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.572875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.575223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.602097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 976/1968] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:01 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:01.606296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.616552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e51a8-f7dc-462d-be76-cb0921002a73 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.621150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.621767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.622050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.622322 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.623005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.624599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.638289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 993/1969] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:01.642758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.652767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34335-b9b8-400a-a7ce-9e71e21a97d5 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.657502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.658165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.658484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.658761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] PATH_INFO: `/v3/projects/5923fedbf3144a0d919b094a435ecbc0/users/e36cc24d08534041aef611c0ea976fdc/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.673542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5923fedbf3144a0d919b094a435ecbc0, user_id=e36cc24d08534041aef611c0ea976fdc, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.676277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.693604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 977/1970] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:01 2022] PUT /identity/v3/projects/5923fedbf3144a0d919b094a435ecbc0/users/e36cc24d08534041aef611c0ea976fdc/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:01.697931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.708162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1ed2e-8ed6-4d3a-bd3e-16f8431b7cc1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.770171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.770839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.771125 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.771400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.772062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.773673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0558fb-f884-41cd-b582-a962314192e6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.785243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 994/1971] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:01.789594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.812972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56b3fd-08b4-4d18-92b1-245b40d03943 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.819142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.819530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.819897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.820170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] PATH_INFO: `/v3/projects/5923fedbf3144a0d919b094a435ecbc0/users/e36cc24d08534041aef611c0ea976fdc/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.822516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5923fedbf3144a0d919b094a435ecbc0, user_id=e36cc24d08534041aef611c0ea976fdc, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.825373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.839747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 978/1972] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:01 2022] PUT /identity/v3/projects/5923fedbf3144a0d919b094a435ecbc0/users/e36cc24d08534041aef611c0ea976fdc/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:01.852664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.853446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.854132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.892240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] MFA Rules not processed for user `e36cc24d08534041aef611c0ea976fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:01.893503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:01.989379 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:02.008286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 995/1973] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:02.022785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:02.024936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:02.037577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:02.166721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d283a-6651-4b0d-a8da-96d967fb266a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:02.170844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:02.171730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:02.172180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:02.172572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:02.184707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:02.185791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:02.186384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:02.187288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:02.201308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e784a-4001-4104-83d9-a0036f432ef5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:02.202041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 979/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:02.205929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:02.206771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:02.207533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:02.207816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:02.209651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:02.291336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:02.292804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d84953-3e8b-4020-92b8-885e6c8559ba service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:02.304064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 996/1975] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:02.979226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:03.053907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372306f7-7377-4081-a432-e43c57c7e295 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:03.060803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:03.061462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:03.061775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:03.061981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] PATH_INFO: `/v3/users/c1c26ee1c88940c5bf3d503b7aaf91d3` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:03.063423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c26ee1c88940c5bf3d503b7aaf91d3)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:03.065355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:03.112443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c26ee1c88940c5bf3d503b7aaf91d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.125083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c26ee1c88940c5bf3d503b7aaf91d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.153183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c26ee1c88940c5bf3d503b7aaf91d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.153957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c26ee1c88940c5bf3d503b7aaf91d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.162412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c26ee1c88940c5bf3d503b7aaf91d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.172632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 980/1976] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:02 2022] DELETE /identity/v3/users/c1c26ee1c88940c5bf3d503b7aaf91d3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:03.174289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:03.179693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:03.179693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:03.203920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:03.208125 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:03.296770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 997/1977] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:03.310522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:03.326397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:03.374177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:03.406615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232abe-d613-4234-9f54-efc05da5cd7f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:03.410993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:03.411624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:03.411949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:03.412212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:03.421969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:03.423417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:03.435364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 981/1978] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:03.736753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:03.738232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:03.756164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00a094-3e07-4e7a-b1fe-addba5277066 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:03.761226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:03.762006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:03.762329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:03.762637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] PATH_INFO: `/v3/projects/390f81ebb576487d90378d341336ec3f` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:03.763847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=390f81ebb576487d90378d341336ec3f)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:03.765842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:03.826571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390f81ebb576487d90378d341336ec3f', 'request_id': 'req-824b2946-f53d-4798-af4f-1a88de133436'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.832658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390f81ebb576487d90378d341336ec3f', 'request_id': 'req-824b2946-f53d-4798-af4f-1a88de133436'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.839256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390f81ebb576487d90378d341336ec3f', 'request_id': 'req-824b2946-f53d-4798-af4f-1a88de133436'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.856827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] The token cache is being invalidate because project 390f81ebb576487d90378d341336ec3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:03.857123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.857834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:03.859411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 998/1979] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:03 2022] DELETE /identity/v3/projects/390f81ebb576487d90378d341336ec3f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:03.863464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:03.874899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58e379-02b1-4d26-a49b-fb75d69d73a5 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:03.942489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:03.943173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:03.943449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:03.943716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] PATH_INFO: `/v3/users/e36cc24d08534041aef611c0ea976fdc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:03.944836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e36cc24d08534041aef611c0ea976fdc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:03.946814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.008485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e36cc24d08534041aef611c0ea976fdc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.018957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e36cc24d08534041aef611c0ea976fdc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.034997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e36cc24d08534041aef611c0ea976fdc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.036071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e36cc24d08534041aef611c0ea976fdc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.045610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5626f188-b8e3-4437-8b99-8a379ef0a5ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e36cc24d08534041aef611c0ea976fdc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.053713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 982/1980] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:03 2022] DELETE /identity/v3/users/e36cc24d08534041aef611c0ea976fdc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:04.380458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.383239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:04.405622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b2946-f53d-4798-af4f-1a88de133436 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.424030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:04.500296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.501429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.501826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.502108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] PATH_INFO: `/v3/projects/5923fedbf3144a0d919b094a435ecbc0` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.503790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5923fedbf3144a0d919b094a435ecbc0)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.506356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.562961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5923fedbf3144a0d919b094a435ecbc0', 'request_id': 'req-4d2f8722-2643-4f18-a038-1859c94bac82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.568712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5923fedbf3144a0d919b094a435ecbc0', 'request_id': 'req-4d2f8722-2643-4f18-a038-1859c94bac82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.575080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5923fedbf3144a0d919b094a435ecbc0', 'request_id': 'req-4d2f8722-2643-4f18-a038-1859c94bac82'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.589076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] The token cache is being invalidate because project 5923fedbf3144a0d919b094a435ecbc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:04.589363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.589993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4d2f8722-2643-4f18-a038-1859c94bac82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:04.591566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 999/1981] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:04 2022] DELETE /identity/v3/projects/5923fedbf3144a0d919b094a435ecbc0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:04.597546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.598025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.598116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.619249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:04.620621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:04.671493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:04.692809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 983/1982] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:04.701802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.702093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.702361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.723706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:04.725077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:04.733763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1000/1983] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:04.739154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.749324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928dcf64-6eb8-4601-8414-4c992a7da348 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.754265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.754897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.755202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.755484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.756345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:04.756455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.758073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.771474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 984/1984] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:04.775726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.785127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f6c44-9e2c-444f-a967-582c3b59404f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.790418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.791059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.791337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.791607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.792287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.794327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.826770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1001/1985] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:04 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:04.831145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.842769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd5203-12fe-469c-95d0-9d59b1e71012 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.912075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.912495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.912776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.913136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.913822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.915889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.938219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 985/1986] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:04 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:04.942491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.953966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf8408-2224-40c1-a241-b37c81c6335b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.959190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.959855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.960137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.960409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.961062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.962689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:04.973701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1002/1987] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:04.977738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:04.988523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bce864-5076-4d84-a224-ba23f22cc90a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:04.993546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:04.994165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:04.994485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:04.994797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:04.995455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:04.997190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257083-1e47-4fe1-82f8-3790f1e4a36e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.008167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 986/1988] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:05.012462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.022962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669fd22-e03b-46fe-9f20-32060fe932b6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.028716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.029611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.029984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.030345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] PATH_INFO: `/v3/projects/0df0d72f30114e0ca8be04d1a20c47e8/users/2474c44938774c4bbf7155ff544fa139/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.048859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df0d72f30114e0ca8be04d1a20c47e8, user_id=2474c44938774c4bbf7155ff544fa139, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.051603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.067751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1003/1989] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:05 2022] PUT /identity/v3/projects/0df0d72f30114e0ca8be04d1a20c47e8/users/2474c44938774c4bbf7155ff544fa139/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:05.078458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.078741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.078990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.110282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] MFA Rules not processed for user `2474c44938774c4bbf7155ff544fa139`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:05.111577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:05.183178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 987/1990] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:05.187900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.232996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:05.257779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d185f53-6482-4a06-b435-da9f9bb91f5f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.263177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.263802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.264084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.264354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.265004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.267066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.301818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1004/1991] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:05 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:05.305881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.317700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f340c23-1d93-4c8d-a0bd-d9c96c47d5e1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.367635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:05.390714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.391336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.391614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.391888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.392545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.394562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.419929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 988/1992] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:05 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:05.424084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.435256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037d04c-5ff9-4286-8018-105d5f7da511 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.440599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.441880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.442157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.442499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.443100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.444726 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.455750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1005/1993] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:05.460110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.470785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15831b80-d0d5-4006-a076-03d64cb9d86c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.476365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.477019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.477293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.477558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] PATH_INFO: `/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca/users/d19bc68b8f6c42318e1597d25b414d71/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.492778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb86cfe710134fbeb6fa4dab6945a3ca, user_id=d19bc68b8f6c42318e1597d25b414d71, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.495460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.509570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 989/1994] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:05 2022] PUT /identity/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca/users/d19bc68b8f6c42318e1597d25b414d71/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:05.513970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.523765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d0c0a-b05e-46ba-8dcc-67df101b5d6a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.588724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.589636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.589879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.590150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.590872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.592849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee569c86-6c6e-40df-9334-728a37b03706 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.603886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1006/1995] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:05.608288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.624641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1608b7-636b-426b-aed4-91b3e0abcf82 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.642594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.644050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.646482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.646895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] PATH_INFO: `/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca/users/d19bc68b8f6c42318e1597d25b414d71/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.649643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb86cfe710134fbeb6fa4dab6945a3ca, user_id=d19bc68b8f6c42318e1597d25b414d71, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.652543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.675147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 990/1996] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:05 2022] PUT /identity/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca/users/d19bc68b8f6c42318e1597d25b414d71/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:05.685363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.685664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.685835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.723444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] MFA Rules not processed for user `d19bc68b8f6c42318e1597d25b414d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:05.724629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:05.801241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1007/1997] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:05.815392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:05.817117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:05.876648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a0177-787c-4b31-8818-b00bb5a7e73d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:05.880425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:05.881039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:05.881318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:05.881592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:05.891093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:05.892547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:05.905119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 991/1998] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:05.992437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:06.005585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223432-3d7e-4d5a-b858-95f5e11d278f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:06.010511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:06.011129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:06.011402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:06.011646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:06.044376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:06.081982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:06.083466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7c41-89d1-4268-8858-210e60b58cb0 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:06.093889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1008/1999] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:06.435857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:06.497147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2c2c-12ca-4c6b-a52b-7b11d0c34858 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:06.502615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:06.503269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:06.503550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:06.503878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] PATH_INFO: `/v3/users/2474c44938774c4bbf7155ff544fa139` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:06.505035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2474c44938774c4bbf7155ff544fa139)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:06.507181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:06.556810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:06.564942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2474c44938774c4bbf7155ff544fa139'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:06.588299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2474c44938774c4bbf7155ff544fa139'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:06.603406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2474c44938774c4bbf7155ff544fa139'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:06.604102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2474c44938774c4bbf7155ff544fa139'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:06.611080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2474c44938774c4bbf7155ff544fa139'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:06.617051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 992/2000] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:06 2022] DELETE /identity/v3/users/2474c44938774c4bbf7155ff544fa139 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:06.621614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:06.622056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:06.622145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:06.642794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:06.644013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:06.714265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1009/2001] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:06.721571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:06.723301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:06.826486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdc14b-6a37-4541-b9be-125641fa40fc admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:06.830518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:06.831232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:06.831518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:06.831874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:06.842431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:06.844265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:06.855313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 993/2002] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:07.231749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:07.233431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:07.250755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e920b-29ce-4cf3-bf95-19d2b73ec211 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:07.257088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:07.258136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:07.258561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:07.258902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] PATH_INFO: `/v3/projects/0df0d72f30114e0ca8be04d1a20c47e8` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:07.260219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df0d72f30114e0ca8be04d1a20c47e8)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:07.262609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:07.318722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df0d72f30114e0ca8be04d1a20c47e8', 'request_id': 'req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.324671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df0d72f30114e0ca8be04d1a20c47e8', 'request_id': 'req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.331158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df0d72f30114e0ca8be04d1a20c47e8', 'request_id': 'req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.346688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] The token cache is being invalidate because project 0df0d72f30114e0ca8be04d1a20c47e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:07.347058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.347715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.349356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1010/2003] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:07 2022] DELETE /identity/v3/projects/0df0d72f30114e0ca8be04d1a20c47e8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:07.353668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:07.364049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d576d7b-29f9-41a8-bdc7-81d9fb6ffa14 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:07.424731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:07.425444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:07.425757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:07.426034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] PATH_INFO: `/v3/users/d19bc68b8f6c42318e1597d25b414d71` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:07.427269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19bc68b8f6c42318e1597d25b414d71)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:07.429238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:07.487327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19bc68b8f6c42318e1597d25b414d71'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.496311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19bc68b8f6c42318e1597d25b414d71'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.504046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:07.512587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19bc68b8f6c42318e1597d25b414d71'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.513859 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19bc68b8f6c42318e1597d25b414d71'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.521374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-54cd19bd-d718-41fd-803d-1e8c83569cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19bc68b8f6c42318e1597d25b414d71'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.528629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 994/2004] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:07 2022] DELETE /identity/v3/users/d19bc68b8f6c42318e1597d25b414d71 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:07.816024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:07.817660 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:07.835010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735626-58a4-4c8d-8b7d-7cbeb8d3801f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:07.886753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:07.894680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:07.895331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:07.895963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:07.896321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] PATH_INFO: `/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:07.897506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb86cfe710134fbeb6fa4dab6945a3ca)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:07.899598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:07.953951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb86cfe710134fbeb6fa4dab6945a3ca', 'request_id': 'req-b2f99286-55aa-4054-a02a-b0dc47756ac3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.959654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb86cfe710134fbeb6fa4dab6945a3ca', 'request_id': 'req-b2f99286-55aa-4054-a02a-b0dc47756ac3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.966781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb86cfe710134fbeb6fa4dab6945a3ca', 'request_id': 'req-b2f99286-55aa-4054-a02a-b0dc47756ac3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.984802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] The token cache is being invalidate because project eb86cfe710134fbeb6fa4dab6945a3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:07.985037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.987475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b2f99286-55aa-4054-a02a-b0dc47756ac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:07.989224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1011/2005] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:07 2022] DELETE /identity/v3/projects/eb86cfe710134fbeb6fa4dab6945a3ca => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:07.993689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:07.994791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:07.995191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.023017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:08.024292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:08.090847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 995/2006] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:08.099120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.099375 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.099929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.123150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:08.125254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:08.136340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1012/2007] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:08.141404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.151395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5d36-468d-46cb-8a72-e150e7061cf2 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.156905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.157715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.158140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.158455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.159120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:08.159428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.161046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.180012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 996/2008] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:08.182221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.191510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33930b82-6889-435b-a01c-b5678230ced2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.196186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.196850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.197127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.197452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.198031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.200003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.230410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1013/2009] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:08.232770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.243041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90317f-21e7-46ae-bd11-e56f45a83f46 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.305090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.305736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.306001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.306282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.307028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.309048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.333637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 997/2010] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:08 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:08.338679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.351505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd026-0007-4326-9d5a-438d506ae210 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.357215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.357845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.358121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.358425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.359090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.360733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.373031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1014/2011] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:08.378017 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.388998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b11f-9b88-4475-818e-37ff27f78c58 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.394577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.395317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.395683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.395958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.396808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.398455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb1e6-7332-4c87-87bb-dfce60db6e98 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.410195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 998/2012] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:08.414806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.425421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeeb6b-2c9f-43c1-af2a-f5276cde22fe admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.430634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.431324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.431605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.431880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] PATH_INFO: `/v3/projects/3e09659f941f4e2daf2879c4385a1c7a/users/c7ac00c97ad0472b96d807ef6e524083/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.447316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e09659f941f4e2daf2879c4385a1c7a, user_id=c7ac00c97ad0472b96d807ef6e524083, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.449993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.464362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1015/2013] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:08 2022] PUT /identity/v3/projects/3e09659f941f4e2daf2879c4385a1c7a/users/c7ac00c97ad0472b96d807ef6e524083/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:08.475322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.475623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.475882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.507948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] MFA Rules not processed for user `c7ac00c97ad0472b96d807ef6e524083`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:08.509233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:08.549126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:08.584104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 999/2014] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:08.596006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.597958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:08.662002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47623279-6a1d-4872-8133-ebb1dfb1944f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.665712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.666337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.666647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.666914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.675886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.677347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ab959-d160-44ac-a406-277b43cf7560 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.690889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1016/2015] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:08.710828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.778013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065b85-b895-4bf0-b829-ebd336ab498b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:08.782843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:08.783504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.783938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.784215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] PATH_INFO: `/v3/users/c7ac00c97ad0472b96d807ef6e524083` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.785372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ac00c97ad0472b96d807ef6e524083)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:08.787371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:08.847712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ac00c97ad0472b96d807ef6e524083'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:08.857807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ac00c97ad0472b96d807ef6e524083'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:08.872982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ac00c97ad0472b96d807ef6e524083'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:08.873972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ac00c97ad0472b96d807ef6e524083'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:08.882519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ac00c97ad0472b96d807ef6e524083'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:08.889082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1000/2016] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:08 2022] DELETE /identity/v3/users/c7ac00c97ad0472b96d807ef6e524083 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:08.893965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:08.894537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:08.894624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:08.914999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:08.916343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:08.935836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:08.982065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1017/2017] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:08.992099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:08.993560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:09.059043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988fcc2-a1bf-4b1c-a028-c15c6691279c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:09.063061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:09.063723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.063997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.064277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.074803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:09.076328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f555fc-fd76-4227-b045-8005a143d0eb service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:09.080946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:09.089218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1001/2018] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:09.481725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:09.484270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:09.509299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8160e-2423-402d-9432-6c50384f3f74 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:09.515219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:09.516245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.516682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.517117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] PATH_INFO: `/v3/projects/3e09659f941f4e2daf2879c4385a1c7a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.518561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e09659f941f4e2daf2879c4385a1c7a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:09.521589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:09.586728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e09659f941f4e2daf2879c4385a1c7a', 'request_id': 'req-109cbe7b-bb95-458f-9d01-be19785ae8bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:09.594085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e09659f941f4e2daf2879c4385a1c7a', 'request_id': 'req-109cbe7b-bb95-458f-9d01-be19785ae8bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:09.602230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e09659f941f4e2daf2879c4385a1c7a', 'request_id': 'req-109cbe7b-bb95-458f-9d01-be19785ae8bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:09.624967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] The token cache is being invalidate because project 3e09659f941f4e2daf2879c4385a1c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:09.625337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:09.625979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-109cbe7b-bb95-458f-9d01-be19785ae8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:09.627541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1018/2019] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:09 2022] DELETE /identity/v3/projects/3e09659f941f4e2daf2879c4385a1c7a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:09.636550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.636840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.637117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.660947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:09.662247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:09.730944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1002/2020] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:09.739137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.739419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.739774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.762837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:09.764083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:09.772469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1019/2021] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:09.777081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:09.789040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c728243-4795-4199-b11e-e2f0075fd067 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:09.795175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:09.795802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.796082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.796368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.797061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:09.797367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:09.799106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:09.813662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1003/2022] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:09.818984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:09.835857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10499c3-2962-4d55-bab2-685e06d15c91 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:09.844869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:09.845491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.845767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.846057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.846718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:09.848637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:09.884699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1020/2023] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:09 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:09.889065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:09.899683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774087c-fe92-413d-afbe-b388659c9ab9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:09.967505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:09.968753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:09.969170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:09.969534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:09.971008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:09.974835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:10.000109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1004/2024] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:09 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:10.004216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:10.014769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ed403-c8f6-4c3d-9c37-ae60da498894 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:10.019563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:10.020203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:10.020493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:10.020773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:10.021424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:10.023088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:10.034191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1021/2025] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:10.039803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:10.050295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507d88c-d4bf-49fb-bccd-6e3c5db073d4 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:10.055328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:10.055976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:10.056259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:10.056529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:10.057194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:10.059016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c97e7e-c787-425f-a3a3-7f9eebd53c1d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:10.071006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1005/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:10.075401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:10.087040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfa90b-60a2-4411-8564-eb6ca76fb3ce admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:10.092392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:10.093081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:10.093360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:10.093658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] PATH_INFO: `/v3/projects/e0a89bb1e9c24a3d8ee45633959e59ea/users/7c17f29c73f643ca83a08f4e44e6279b/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:10.111552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a89bb1e9c24a3d8ee45633959e59ea, user_id=7c17f29c73f643ca83a08f4e44e6279b, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:10.114606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:10.134007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1022/2027] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:10 2022] PUT /identity/v3/projects/e0a89bb1e9c24a3d8ee45633959e59ea/users/7c17f29c73f643ca83a08f4e44e6279b/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:10.140435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27688dd-dfd4-410b-8543-a56795a00a5e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:10.140770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27688dd-dfd4-410b-8543-a56795a00a5e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:10.141076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27688dd-dfd4-410b-8543-a56795a00a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:10.160760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d27688dd-dfd4-410b-8543-a56795a00a5e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:10.161981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d27688dd-dfd4-410b-8543-a56795a00a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:10.231432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1006/2028] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:10.242193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:10.244149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:10.298387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:10.313089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dab4a-7e6e-43c1-8187-95a8cccde533 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:10.317134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:10.317762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:10.318048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:10.318316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:10.328818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:10.330214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:10.340120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1023/2029] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:11.119397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc3e11-354b-44c4-bb26-29d456a3dffe None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:11.120078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc3e11-354b-44c4-bb26-29d456a3dffe None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:11.120078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc3e11-354b-44c4-bb26-29d456a3dffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:11.128729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:11.153888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-abcc3e11-354b-44c4-bb26-29d456a3dffe None None] MFA Rules not processed for user `7c17f29c73f643ca83a08f4e44e6279b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:11.155707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-abcc3e11-354b-44c4-bb26-29d456a3dffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:11.232035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1007/2030] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:11.244328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:11.245878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:11.310949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e7e91-8759-43d8-aaed-57d59a14b383 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:11.314945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:11.315571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:11.315843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:11.316203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:11.324987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:11.326509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:11.336017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1024/2031] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:12.746953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:12.747555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:12.747555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:12.768916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] MFA Rules not processed for user `7c17f29c73f643ca83a08f4e44e6279b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:12.770194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:12.778505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1008/2032] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:13.855556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:13.857713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:13.874062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:13.876087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:13.918650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0d72-9331-4820-8d15-25d85afba838 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:13.920817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399f1c8-fc63-4c93-a0ba-40c56ddf2e36 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:13.928170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:13.929435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:13.929709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:13.930497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] PATH_INFO: `/v3/users/a8cebef35be6496ba7735ca208925e8f` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:13.932564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8cebef35be6496ba7735ca208925e8f)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:13.932968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:13.936136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:13.936727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:13.938279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:13.940162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] PATH_INFO: `/v3/users/4d14eab58e924a5db5eca2f2a43c4a36` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:13.942693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d14eab58e924a5db5eca2f2a43c4a36)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:13.948116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:14.026032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8cebef35be6496ba7735ca208925e8f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.041474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8cebef35be6496ba7735ca208925e8f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.067171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d14eab58e924a5db5eca2f2a43c4a36'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.072320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8cebef35be6496ba7735ca208925e8f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.075131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8cebef35be6496ba7735ca208925e8f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.087941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d14eab58e924a5db5eca2f2a43c4a36'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.101127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fcf7351a-e010-469f-bcd6-0e35ed0e11e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8cebef35be6496ba7735ca208925e8f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.116749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1009/2033] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:13 2022] DELETE /identity/v3/users/a8cebef35be6496ba7735ca208925e8f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:14.120943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d614e6-4830-4180-9f76-078495937d97 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:14.122330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d614e6-4830-4180-9f76-078495937d97 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:14.123511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d14eab58e924a5db5eca2f2a43c4a36'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.124094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d614e6-4830-4180-9f76-078495937d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:14.125241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d14eab58e924a5db5eca2f2a43c4a36'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.145305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b398c74e-f1db-4b38-b921-cb84c2d0e582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d14eab58e924a5db5eca2f2a43c4a36'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:14.164678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-85d614e6-4830-4180-9f76-078495937d97 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:14.166183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-85d614e6-4830-4180-9f76-078495937d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:14.166433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1025/2034] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:13 2022] DELETE /identity/v3/users/4d14eab58e924a5db5eca2f2a43c4a36 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:14.170911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:14.171202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:14.171478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:14.203624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:14.204702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:14.238777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:14.256350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1010/2035] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:14.264175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d614e6-4830-4180-9f76-078495937d97 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:14.265858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-85d614e6-4830-4180-9f76-078495937d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:14.300885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1026/2036] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:14.310725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:14.313966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:14.322357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:14.403829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d614e6-4830-4180-9f76-078495937d97 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:14.414242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:14.414941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:14.415223 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:14.415498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:14.424773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebf0f8-1788-4a48-83c2-9c481f253628 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:14.443793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:14.444472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:14.444755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:14.445295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:14.450782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:14.452277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:14.457571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:14.459373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:14.466031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1011/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:14.473496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1027/2038] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:14.910252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:14.912179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:14.931946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73e58-c9d7-4a0e-a90f-dc74700a83dc service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:14.938186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:14.939033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:14.939314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:14.939616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] PATH_INFO: `/v3/projects/da9b406b6ac14778ac583ef4c204f588` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:14.940839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9b406b6ac14778ac583ef4c204f588)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:14.943406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.003199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.005231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.014711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9b406b6ac14778ac583ef4c204f588', 'request_id': 'req-9913c0d9-6782-4987-8370-3bb6538a4be3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.021522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081924c-0a32-44ab-8333-d047ff181955 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.022086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9b406b6ac14778ac583ef4c204f588', 'request_id': 'req-9913c0d9-6782-4987-8370-3bb6538a4be3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.030113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9b406b6ac14778ac583ef4c204f588', 'request_id': 'req-9913c0d9-6782-4987-8370-3bb6538a4be3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.045132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] The token cache is being invalidate because project da9b406b6ac14778ac583ef4c204f588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:15.045694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.046087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9913c0d9-6782-4987-8370-3bb6538a4be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.050472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1012/2039] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:14 2022] DELETE /identity/v3/projects/da9b406b6ac14778ac583ef4c204f588 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:15.053988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4844b-bdfe-455f-8d69-db732943d956 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.055138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4844b-bdfe-455f-8d69-db732943d956 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.055358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4844b-bdfe-455f-8d69-db732943d956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.086289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5ad4844b-bdfe-455f-8d69-db732943d956 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:15.087806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5ad4844b-bdfe-455f-8d69-db732943d956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.119266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.119845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.120291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.120575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] PATH_INFO: `/v3/projects/a7e14dcc651e4ca89ae36ba87996b23d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.123034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e14dcc651e4ca89ae36ba87996b23d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.125300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.156488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1013/2040] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:15.165590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.165990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.166297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.183201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e14dcc651e4ca89ae36ba87996b23d', 'request_id': 'req-d8b84ba2-1832-401e-a479-ea8caf256a08'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.190925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:15.192164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.192423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e14dcc651e4ca89ae36ba87996b23d', 'request_id': 'req-d8b84ba2-1832-401e-a479-ea8caf256a08'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.199802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e14dcc651e4ca89ae36ba87996b23d', 'request_id': 'req-d8b84ba2-1832-401e-a479-ea8caf256a08'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.217008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:15.218177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] The token cache is being invalidate because project a7e14dcc651e4ca89ae36ba87996b23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:15.218809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.219647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.221393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1028/2041] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:15 2022] DELETE /identity/v3/projects/a7e14dcc651e4ca89ae36ba87996b23d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:15.225889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.227523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.266560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1014/2042] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:15.268700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:15.271075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.281526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636bc0a-4a2a-41a4-9018-4aed6880384c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.286346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.286992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.287274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.287573 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.288256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:15.288562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.290173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.297836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b84ba2-1832-401e-a479-ea8caf256a08 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.301952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1015/2043] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:15.302751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.303423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.303701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.303973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] PATH_INFO: `/v3/users/f6f526fcddaf4450bf5330b293334c88` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.305121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f526fcddaf4450bf5330b293334c88)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.306224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.307123 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.315804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0926e30-66b4-4af7-a3b5-e8855d56f7ba admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.320590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.321198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.321474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.321749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.322399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.324401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.356468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1016/2044] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:15.356709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f526fcddaf4450bf5330b293334c88'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.357907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.365950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f526fcddaf4450bf5330b293334c88'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.385162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f526fcddaf4450bf5330b293334c88'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.386188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f526fcddaf4450bf5330b293334c88'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.395185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f526fcddaf4450bf5330b293334c88'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.401730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1029/2045] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:15 2022] DELETE /identity/v3/users/f6f526fcddaf4450bf5330b293334c88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:15.426932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7f2c5-b1d4-4dff-8425-e17ca049c14f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.491592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.492139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.492421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.492696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.493345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.495350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.527433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1017/2046] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:15.531713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.533258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.548005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6082c8-9d6d-4a30-9375-038c2422dc7d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.553121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.554529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.554627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.555349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.555558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.557952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.574687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1030/2047] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:15.575821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.588135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33f13-8fc0-47ef-976a-595d27b43b2d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.595795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.596679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.597074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.597416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.598160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.610780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61483f67-1995-4347-b089-2455753764f1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.630719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1018/2048] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:15.632060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.643022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0dc2a-1ec3-496f-9efa-77f1519fc6ed admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.647750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.648453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.648735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.649009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] PATH_INFO: `/v3/projects/36ceef1e42174c70aa5f3fd53d0812c7/users/3756b1fedff545aaaf076d5e01108e7e/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.665949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ceef1e42174c70aa5f3fd53d0812c7, user_id=3756b1fedff545aaaf076d5e01108e7e, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.669349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.692789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1031/2049] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:15 2022] PUT /identity/v3/projects/36ceef1e42174c70aa5f3fd53d0812c7/users/3756b1fedff545aaaf076d5e01108e7e/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:15.703215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.703503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.704249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.733943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] MFA Rules not processed for user `3756b1fedff545aaaf076d5e01108e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:15.735273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.740940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:15.826099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1019/2050] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:15.826244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.827327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:15.827613 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.894635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178301e-72af-4aa6-870e-dcaf83631bf3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.899292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.899980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.900317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.900648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.901387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.903324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080313d7-3f59-458f-92a7-504e57e06483 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:15.903426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.908201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:15.908838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:15.909317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:15.909588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] PATH_INFO: `/v3/projects/7e49b0dda20c4213a85d30c95b88e488` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:15.910762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e49b0dda20c4213a85d30c95b88e488)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:15.912717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:15.938227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1020/2051] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:15.939015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:15.939594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:15.967477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e49b0dda20c4213a85d30c95b88e488', 'request_id': 'req-f35f8b34-7d52-496d-a8dc-16af1a933512'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.973962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e49b0dda20c4213a85d30c95b88e488', 'request_id': 'req-f35f8b34-7d52-496d-a8dc-16af1a933512'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.980381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e49b0dda20c4213a85d30c95b88e488', 'request_id': 'req-f35f8b34-7d52-496d-a8dc-16af1a933512'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.994491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] The token cache is being invalidate because project 7e49b0dda20c4213a85d30c95b88e488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:15.995024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.995612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f35f8b34-7d52-496d-a8dc-16af1a933512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:15.997516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1032/2052] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:15 2022] DELETE /identity/v3/projects/7e49b0dda20c4213a85d30c95b88e488 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:16.002531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba56af8-7664-4e84-8a23-b4aed8638bee None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.003043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba56af8-7664-4e84-8a23-b4aed8638bee None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.003043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba56af8-7664-4e84-8a23-b4aed8638bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.010477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9677466-b366-4380-9271-4333264dac54 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.028238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6ba56af8-7664-4e84-8a23-b4aed8638bee None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:16.029504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6ba56af8-7664-4e84-8a23-b4aed8638bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.061664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:16.076453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.077089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.077368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.077641 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.078446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.080421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.104709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1033/2053] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:16.106053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.106335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.106629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.114412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1021/2054] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:15 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:16.117960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.119834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.128070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:16.129641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.133645 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0cf5c-f8b8-49d4-b322-1c8d8da67945 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.137688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1034/2055] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:16.138568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.139240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.139479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.141174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.141174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.141718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.142398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.150662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc1483-43f5-4602-93ce-e44202574444 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.152997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1022/2056] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:16.155794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.156527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.156805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.157082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.157247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.157734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:16.158037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.159741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.166991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e89fd-920b-4a22-b597-61a768e04e42 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.172764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1035/2057] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:59:16.177101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.181039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.181677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.181965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.182248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.182921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.184540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.187342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09795c6c-aeaf-41dc-afc8-97649faeb551 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.192348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.192961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.193239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.193511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.194144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.196091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.197384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1023/2058] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:16.201512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.212047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6f86-c129-49f8-97fc-60daa8261f1d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.217897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.218676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.218955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.219240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] PATH_INFO: `/v3/projects/e386b689a1fc4f328d1c330fbe30fe42/users/5995fe46e72e401d81ee34d2adb53a5f/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.226206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1036/2059] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:16.230081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.234974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e386b689a1fc4f328d1c330fbe30fe42, user_id=5995fe46e72e401d81ee34d2adb53a5f, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.237649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728548bc-b08f-4c6b-834d-ef505b47669f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.252515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1024/2060] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:16 2022] PUT /identity/v3/projects/e386b689a1fc4f328d1c330fbe30fe42/users/5995fe46e72e401d81ee34d2adb53a5f/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:16.263244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.264056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.264056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.302002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] MFA Rules not processed for user `5995fe46e72e401d81ee34d2adb53a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:16.303016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.304321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7161c-97a6-4aa4-9688-e2c9c7ae77a0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.319365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:16.370273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.370958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.371246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.371519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.372165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.374236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.383327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1025/2061] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:16.387532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.398186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e494a1b-c13b-4fbe-8109-8efbac653008 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.403478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1037/2062] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:16.403614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.403614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.403819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.404002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.404501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.404634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.406648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.414682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0423f-ef11-451c-b042-7f5da6591642 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.419438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.420047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.420311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.420588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.421457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.423159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.438174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1038/2063] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:16.439409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1026/2064] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:16.439520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.440207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.453243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7a01f-a998-4c80-825c-eab963bf7489 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.503243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:16.516085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e70d79-1aa3-442b-8000-f5c3482f6a08 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.521478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.522108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.522479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.522764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.523406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.525427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.527263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.527891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.528167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.528433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.529070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.530718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.541428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1039/2065] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:16.545695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.549518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1027/2066] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:16.553342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.558954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9044370-4c06-424b-9cc4-cb7abeb5c82d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.564083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a244c74-afc9-49c1-b994-2ca4309971a4 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.564540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.565233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.565513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.565783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] PATH_INFO: `/v3/projects/8418613741204d23a4d103bc625e80a0/users/2593941157cd468d9368170604e7a3bc/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.571071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.571633 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.572171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.572453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.573108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.574777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.585215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8418613741204d23a4d103bc625e80a0, user_id=2593941157cd468d9368170604e7a3bc, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.590407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e26585-45c5-4ec4-855f-d3652ef8e481 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.590602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1028/2067] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:16.591739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.602929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302119e-c0ab-481c-94e8-d8edf0dedf42 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.606281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1040/2068] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:16 2022] PUT /identity/v3/projects/8418613741204d23a4d103bc625e80a0/users/2593941157cd468d9368170604e7a3bc/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:16.618079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.618380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.618669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.647538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] MFA Rules not processed for user `2593941157cd468d9368170604e7a3bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:16.648775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.660802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.661656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.661908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.662325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] PATH_INFO: `/v3/projects/d16118d32221404ea023022214f7fa7e/users/7a3fc455e4994ba78546fd79c20fad2b/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.678873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16118d32221404ea023022214f7fa7e, user_id=7a3fc455e4994ba78546fd79c20fad2b, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.681867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.698940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1029/2069] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:16 2022] PUT /identity/v3/projects/d16118d32221404ea023022214f7fa7e/users/7a3fc455e4994ba78546fd79c20fad2b/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:16.702683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.711042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:16.729275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1041/2070] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:16.730722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.743096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab0259-7c48-4772-8b7f-46699a700978 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.788420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34b4eb-b345-4b45-8939-d1436e9c6aa8 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.793240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.793869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.794180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.794588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.795418 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.797124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.817331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1030/2071] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:16.818826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.820474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.821286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.821612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.821947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.822717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.824736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.830595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb188ab-60c1-44a5-83fb-536c8c881ba2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.835788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.836652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.836932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.837703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] PATH_INFO: `/v3/projects/d16118d32221404ea023022214f7fa7e/users/7a3fc455e4994ba78546fd79c20fad2b/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.839847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16118d32221404ea023022214f7fa7e, user_id=7a3fc455e4994ba78546fd79c20fad2b, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.843095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8e1c3-f590-463d-897f-20a3a7a59ae5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.858786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1042/2072] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:16.863086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.864509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1031/2073] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:16 2022] PUT /identity/v3/projects/d16118d32221404ea023022214f7fa7e/users/7a3fc455e4994ba78546fd79c20fad2b/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:16.876133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.876421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.876692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.913387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] MFA Rules not processed for user `7a3fc455e4994ba78546fd79c20fad2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:16.914655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:16.942289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:16.947682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01e7de-a8b6-41e0-a3fe-131a2cb1c63c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:16.953519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:16.954131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:16.954437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:16.954713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:16.955360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:16.957299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:16.985056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1043/2074] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:16.985949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:16.996470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1032/2075] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:16.999471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af7559-e563-4ffd-b7dc-f1688e779856 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.004463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.005091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.005369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.005635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.006283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.008090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.009291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.010985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:17.022405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1044/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:17.024288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.037761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20818090-4e5e-4d28-95ed-1d411c983915 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.046927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.047875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.048334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.048849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] PATH_INFO: `/v3/projects/8ac4b4675bc9442f9e73792e72b67741/users/275862fdd4c749a995bf71684d8005d8/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.069341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac4b4675bc9442f9e73792e72b67741, user_id=275862fdd4c749a995bf71684d8005d8, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.072793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.085199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e447d-ec07-4c7b-b86e-2329d58bee8c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.097209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1045/2077] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:17 2022] PUT /identity/v3/projects/8ac4b4675bc9442f9e73792e72b67741/users/275862fdd4c749a995bf71684d8005d8/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:17.102814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.155145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.155778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.156353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.156656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.158473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:17.158966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:17.185992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0130c2-5f45-49d2-8de3-8dca5defb2ce admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.191115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.193949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.193949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.193949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.193949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.194832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.207098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1046/2078] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:17.211164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.222924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9101c7-2377-43d9-a131-121c67800fde admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.227868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.228702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.229082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.229364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] PATH_INFO: `/v3/projects/8ac4b4675bc9442f9e73792e72b67741/users/275862fdd4c749a995bf71684d8005d8/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.232528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac4b4675bc9442f9e73792e72b67741, user_id=275862fdd4c749a995bf71684d8005d8, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.233039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.234751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471a70d-1fd0-41ab-ad00-832b6fb93f93 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.235277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.245031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1033/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:17.246221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1047/2080] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:17 2022] PUT /identity/v3/projects/8ac4b4675bc9442f9e73792e72b67741/users/275862fdd4c749a995bf71684d8005d8/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:17.257024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.257370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.257705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.288424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] MFA Rules not processed for user `275862fdd4c749a995bf71684d8005d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:17.289674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:17.310388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:17.368936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1034/2081] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:17.422678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.426502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:17.492411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e4a7f-e250-466c-ac60-df7eeb6ed195 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.501436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.502120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.502434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.502705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.514986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.516698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.527649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1048/2082] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:17.622640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:17.647672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2c964-2729-402a-99ba-0504b92d23fc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:17.653406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:17.654776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:17.655286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:17.655832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:17.740108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:17.741726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:17.751962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1035/2083] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:18.991932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:19.002458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9356-e471-43c9-86a8-d662bde9bd09 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:19.006704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:19.007319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:19.007604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:19.007848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:19.093629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:19.095973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:19.122534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1049/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:19.999494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:20.009796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81bf0f-d1dd-48fc-a1c2-e710d5a9a353 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:20.013420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:20.014043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:20.014318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:20.014620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:20.079597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:20.081166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e10a75-8273-4113-a420-55baf9cbebfe service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:20.091900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1036/2085] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:20.779335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:20.847968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15e529-240c-430e-95c4-651392a9fa79 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:20.852943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:20.853668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:20.853945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:20.854215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] PATH_INFO: `/v3/users/2593941157cd468d9368170604e7a3bc` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:20.855418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2593941157cd468d9368170604e7a3bc)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:20.857383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:20.859173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:20.922636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2593941157cd468d9368170604e7a3bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:20.944558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2593941157cd468d9368170604e7a3bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:20.959005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2593941157cd468d9368170604e7a3bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:20.959751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2593941157cd468d9368170604e7a3bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:20.968088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2593941157cd468d9368170604e7a3bc'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:20.974335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1050/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:20 2022] DELETE /identity/v3/users/2593941157cd468d9368170604e7a3bc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:20.978967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b442275-7530-4ad3-971e-f800c868117e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:20.979254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b442275-7530-4ad3-971e-f800c868117e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:20.979553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b442275-7530-4ad3-971e-f800c868117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:20.998524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0b442275-7530-4ad3-971e-f800c868117e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:20.999913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0b442275-7530-4ad3-971e-f800c868117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:21.058121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:21.075055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1037/2087] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:21.085230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:21.086865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:21.148815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30210-3195-4ab3-9c98-58bff8f101c9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:21.153034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:21.153650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:21.153938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:21.154201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:21.163532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:21.164984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:21.174876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1051/2088] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:21.457072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b442275-7530-4ad3-971e-f800c868117e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:21.458617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0b442275-7530-4ad3-971e-f800c868117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:21.471737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b442275-7530-4ad3-971e-f800c868117e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:21.477692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:21.478556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:21.478975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:21.479260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] PATH_INFO: `/v3/projects/8418613741204d23a4d103bc625e80a0` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:21.480811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8418613741204d23a4d103bc625e80a0)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:21.482802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:21.535181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8418613741204d23a4d103bc625e80a0', 'request_id': 'req-63286da3-cd57-4631-bba2-4e699d4372f9'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.541427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8418613741204d23a4d103bc625e80a0', 'request_id': 'req-63286da3-cd57-4631-bba2-4e699d4372f9'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.549298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8418613741204d23a4d103bc625e80a0', 'request_id': 'req-63286da3-cd57-4631-bba2-4e699d4372f9'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.567188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] The token cache is being invalidate because project 8418613741204d23a4d103bc625e80a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:21.567427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.568094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.569853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1038/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:21 2022] DELETE /identity/v3/projects/8418613741204d23a4d103bc625e80a0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:21.580109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:21.597189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ecc0c4-286c-45ed-9dbd-098591489e54 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:21.712567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:21.713705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:21.714180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:21.714507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] PATH_INFO: `/v3/users/275862fdd4c749a995bf71684d8005d8` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:21.717069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275862fdd4c749a995bf71684d8005d8)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:21.720857 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:21.757822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:21.788035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275862fdd4c749a995bf71684d8005d8'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.796313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275862fdd4c749a995bf71684d8005d8'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.809022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275862fdd4c749a995bf71684d8005d8'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.809785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275862fdd4c749a995bf71684d8005d8'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.818082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0225062c-bd0b-4dc0-8754-c3492a6c5f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275862fdd4c749a995bf71684d8005d8'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:21.824945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1052/2090] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:21 2022] DELETE /identity/v3/users/275862fdd4c749a995bf71684d8005d8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:22.131949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.133513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:22.147661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63286da3-cd57-4631-bba2-4e699d4372f9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.208353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.208991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.209276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.209582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] PATH_INFO: `/v3/projects/8ac4b4675bc9442f9e73792e72b67741` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.210692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac4b4675bc9442f9e73792e72b67741)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.212597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.266708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac4b4675bc9442f9e73792e72b67741', 'request_id': 'req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:22.272401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac4b4675bc9442f9e73792e72b67741', 'request_id': 'req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:22.278925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac4b4675bc9442f9e73792e72b67741', 'request_id': 'req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:22.292149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] The token cache is being invalidate because project 8ac4b4675bc9442f9e73792e72b67741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:22.292437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:22.293046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-66b5c9d2-8e93-4369-95fd-b2ef9b224b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:22.294693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1039/2091] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:22 2022] DELETE /identity/v3/projects/8ac4b4675bc9442f9e73792e72b67741 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:22.299709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89174d-9b88-4055-9918-413314d46842 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.300012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89174d-9b88-4055-9918-413314d46842 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.300272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89174d-9b88-4055-9918-413314d46842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.321269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2a89174d-9b88-4055-9918-413314d46842 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:22.322592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2a89174d-9b88-4055-9918-413314d46842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:22.397528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1053/2092] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:22.405496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.405852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.406134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.414228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:22.434888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:22.436153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:22.445040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1040/2093] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:22.449725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89174d-9b88-4055-9918-413314d46842 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.463102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89174d-9b88-4055-9918-413314d46842 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.468450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.469128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.469404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.469670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.470317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:22.470642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.472285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.483817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1054/2094] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:22.488317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.498254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62da2-ae20-42de-ba16-5657860258f3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.503635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.504237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.504511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.504779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.505419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.507364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.539128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1041/2095] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:22.543369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.559174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d94e-ef82-4840-bb7c-43f140c4b33e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.623583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.624235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.624591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.625182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.625713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.627775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.652584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1055/2096] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:22.656979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.671348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec4cf4-72b8-404b-995a-37bb056591b5 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.676851 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.677496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.677778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.678050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.678755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.680681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.691890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1042/2097] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:22.696169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.708864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3700c31-4aaf-49f5-9562-3e8f02ef7ac2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.719030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.720394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.721003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.721561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.723093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.726752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7586b41-5ca3-4fd0-b4e6-a8f9a4c4caca admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.745980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1056/2098] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:22.751083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:22.767127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64253735-027b-4762-9661-410ae5f3d7ca admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:22.773371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:22.774060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.774336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.774634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] PATH_INFO: `/v3/projects/ba045cda31eb4c0db873a74df91ee00d/users/de1e14243872408fb95d217220f5063e/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.793052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba045cda31eb4c0db873a74df91ee00d, user_id=de1e14243872408fb95d217220f5063e, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:22.794505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:22.814824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1043/2099] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:22 2022] PUT /identity/v3/projects/ba045cda31eb4c0db873a74df91ee00d/users/de1e14243872408fb95d217220f5063e/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:22.826501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:22.827039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:22.827119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:22.842052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:22.859970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] MFA Rules not processed for user `de1e14243872408fb95d217220f5063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:22.861922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:22.952337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1057/2100] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:22.954301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.022359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111709-5f00-4242-b241-011db2f42f4b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.027267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.027887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.028166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.028473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.029040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.031047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.062740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1044/2101] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:22 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:23.067403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.084297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da7d30-3f19-4210-8f5e-017a08681286 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.151419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.152048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.152362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.152634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.153289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.155289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.182169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1058/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:23 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:23.186642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.199885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2971d-f304-4cc5-a15d-4136633d57c0 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.204608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.205236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.205503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.205780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.206467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.208087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.221592 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1045/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:23.225922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.236404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808dc4-d422-4130-983b-87c265ee8330 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.241506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.242189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.242475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.242753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] PATH_INFO: `/v3/projects/52a4b1078ba84697b286326f3a030401/users/ff5c1930bb934cb684133e847456adc3/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.259685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a4b1078ba84697b286326f3a030401, user_id=ff5c1930bb934cb684133e847456adc3, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.262393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.277142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1059/2104] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:23 2022] PUT /identity/v3/projects/52a4b1078ba84697b286326f3a030401/users/ff5c1930bb934cb684133e847456adc3/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:23.281601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.292584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c12c7b-de22-4c70-87ef-29d9a7abb1ee admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.355123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.355751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.356034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.356309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.356965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.358631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4f921-fccc-45bf-82cd-be954b2518e9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.372749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1046/2105] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:23.373799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.385396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c74ce3-c0c4-40e3-be4f-2f4ec5ac93c6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.391178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.391812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.392086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.392409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] PATH_INFO: `/v3/projects/52a4b1078ba84697b286326f3a030401/users/ff5c1930bb934cb684133e847456adc3/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.394350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a4b1078ba84697b286326f3a030401, user_id=ff5c1930bb934cb684133e847456adc3, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.397003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.408360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1060/2106] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:23 2022] PUT /identity/v3/projects/52a4b1078ba84697b286326f3a030401/users/ff5c1930bb934cb684133e847456adc3/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:23.418829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.419113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.419392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.439902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:23.453089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] MFA Rules not processed for user `ff5c1930bb934cb684133e847456adc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:23.454900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:23.532769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1047/2107] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:23.545154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:23.547284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:23.613509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:23.617594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98478c34-cc68-4e8f-bb02-2efa60f1bba7 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:23.623107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:23.623730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:23.624141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:23.624697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:23.701060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:23.702569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:23.712233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1061/2108] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:30.934490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:30.944481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40feb4-647b-4a84-a499-9ea4c3ef69d8 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:30.953132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:30.953754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:30.954027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:30.954300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:30.963998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:30.965974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e390e95-8eb3-44da-944f-4fd9d0733645 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:30.975750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1048/2109] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:31.555897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:31.557494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:31.660358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33e6d-4db2-42b5-96b0-3d82918c8fd2 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:31.664867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:31.665513 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:31.665785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:31.666051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] PATH_INFO: `/v3/users/3756b1fedff545aaaf076d5e01108e7e` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:31.667194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3756b1fedff545aaaf076d5e01108e7e)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:31.669153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:31.718804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3756b1fedff545aaaf076d5e01108e7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:31.729616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3756b1fedff545aaaf076d5e01108e7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:31.743573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3756b1fedff545aaaf076d5e01108e7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:31.744275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3756b1fedff545aaaf076d5e01108e7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:31.752895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3756b1fedff545aaaf076d5e01108e7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:31.764723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1062/2110] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:31 2022] DELETE /identity/v3/users/3756b1fedff545aaaf076d5e01108e7e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:31.769928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:31.770498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:31.770954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:31.792577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:31.793853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:31.862875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1049/2111] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:31.872801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:31.874402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:31.902881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:31.947376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18aaf-b914-4743-b2f6-ee2435352260 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:31.954415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:31.955741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:31.956522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:31.956941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:31.972492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:31.981639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:31.990996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1063/2112] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:32.544970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:32.550837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:32.571609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70ff7b-01e1-44d7-9950-1dd0b80bac11 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:32.581104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:32.581847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:32.582304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:32.582803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] PATH_INFO: `/v3/projects/36ceef1e42174c70aa5f3fd53d0812c7` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:32.584009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ceef1e42174c70aa5f3fd53d0812c7)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:32.586434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:32.657878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ceef1e42174c70aa5f3fd53d0812c7', 'request_id': 'req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.665094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ceef1e42174c70aa5f3fd53d0812c7', 'request_id': 'req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.673216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ceef1e42174c70aa5f3fd53d0812c7', 'request_id': 'req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.688388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] The token cache is being invalidate because project 36ceef1e42174c70aa5f3fd53d0812c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:32.688623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.689314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.691124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1050/2113] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:32 2022] DELETE /identity/v3/projects/36ceef1e42174c70aa5f3fd53d0812c7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:32.695507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:32.708731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fbff5-01d5-4ea8-b922-4337068f68e6 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:32.820765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:32.821874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:32.822420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:32.823669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] PATH_INFO: `/v3/users/5995fe46e72e401d81ee34d2adb53a5f` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:32.825231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5995fe46e72e401d81ee34d2adb53a5f)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:32.830855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:32.905554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5995fe46e72e401d81ee34d2adb53a5f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.925014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5995fe46e72e401d81ee34d2adb53a5f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.936577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5995fe46e72e401d81ee34d2adb53a5f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.937606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5995fe46e72e401d81ee34d2adb53a5f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.952796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5995fe46e72e401d81ee34d2adb53a5f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:32.963406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1064/2114] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:32 2022] DELETE /identity/v3/users/5995fe46e72e401d81ee34d2adb53a5f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:33.631108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:33.633201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:33.643946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:33.648497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa1e11-5a2e-4aa0-8a34-80be6ba5067c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:33.711771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:33.712426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:33.712716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:33.712983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] PATH_INFO: `/v3/projects/e386b689a1fc4f328d1c330fbe30fe42` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:33.714148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e386b689a1fc4f328d1c330fbe30fe42)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:33.716466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:33.767408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e386b689a1fc4f328d1c330fbe30fe42', 'request_id': 'req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:33.773171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e386b689a1fc4f328d1c330fbe30fe42', 'request_id': 'req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:33.779657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e386b689a1fc4f328d1c330fbe30fe42', 'request_id': 'req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:33.798763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] The token cache is being invalidate because project e386b689a1fc4f328d1c330fbe30fe42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:33.799705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:33.800024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:33.802002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1051/2115] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:33 2022] DELETE /identity/v3/projects/e386b689a1fc4f328d1c330fbe30fe42 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:33.806070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:33.808198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:33.821824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52d4ca-526b-4e05-b77b-46099ab893ab admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:33.884040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:33.925720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:33.927030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:33.927462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:33.928623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] PATH_INFO: `/v3/users/7a3fc455e4994ba78546fd79c20fad2b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:33.930059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3fc455e4994ba78546fd79c20fad2b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:33.942612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:34.011212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3fc455e4994ba78546fd79c20fad2b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.021426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3fc455e4994ba78546fd79c20fad2b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.044722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3fc455e4994ba78546fd79c20fad2b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.045531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3fc455e4994ba78546fd79c20fad2b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.075358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-2c353f32-f662-438d-9a68-2bb37d1984b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3fc455e4994ba78546fd79c20fad2b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.087619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1065/2116] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:33 2022] DELETE /identity/v3/users/7a3fc455e4994ba78546fd79c20fad2b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:34.650144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:34.651720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:34.671867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83737cb8-8ecc-4ca0-b089-f4aca1f6d363 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:34.764176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:34.764825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:34.765103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:34.765599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] PATH_INFO: `/v3/projects/d16118d32221404ea023022214f7fa7e` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:34.766797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16118d32221404ea023022214f7fa7e)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:34.768722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:34.821072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:34.832527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16118d32221404ea023022214f7fa7e', 'request_id': 'req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.840566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16118d32221404ea023022214f7fa7e', 'request_id': 'req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.847809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16118d32221404ea023022214f7fa7e', 'request_id': 'req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.863467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] The token cache is being invalidate because project d16118d32221404ea023022214f7fa7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:34.863795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.864594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-37a2fd6d-c4cb-4743-9e3a-10dd75887f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:34.866409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1052/2117] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:34 2022] DELETE /identity/v3/projects/d16118d32221404ea023022214f7fa7e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:34.871624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:34.872281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:34.872281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:34.895345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:34.896601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:34.968777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1066/2118] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:34.977582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:34.977876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:34.978239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.003619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:35.004972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.013689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1053/2119] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:35.018393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.032503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f920a9-6531-4c90-ae7d-fa4cb5b138f4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.042035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.043162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.043665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.044143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.045346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:35.045897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.049522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.073649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1067/2120] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:35.080016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.099528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57854f11-21ee-4f7e-9d22-0f469e92f1e6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.106601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.107294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.107691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.107994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.108770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.111208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.144007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1054/2121] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:35 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:35.148944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.162404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9793565-8e9b-4a28-bf66-b05987bd53fa admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.220174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:35.233039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.234168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.235091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.235437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.236234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.238524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.263021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1068/2122] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:35 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:35.267395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.279933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187543cd-6615-4d6a-bf5c-d46b36e60338 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.285275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.285874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.286150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.286459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.287119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.288847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.302029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1055/2123] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:35.306610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.319131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6600d50-d84d-48cb-8a4d-97278320c17e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.325157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.325845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.326203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.326593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.327416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.329223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d02a5-2f7d-48d7-96da-57b729dd15e5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.341092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1069/2124] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:35.345891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.356284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334332f9-3500-40ef-b9fe-5e5b3db40e41 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.361521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.362300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.362694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.363049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] PATH_INFO: `/v3/projects/a747512840e14dfca8bf507e70623fa5/users/b5b22eed68ce4d879de69b94084f1fb8/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.383460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a747512840e14dfca8bf507e70623fa5, user_id=b5b22eed68ce4d879de69b94084f1fb8, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.386642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.401329 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1056/2125] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:35 2022] PUT /identity/v3/projects/a747512840e14dfca8bf507e70623fa5/users/b5b22eed68ce4d879de69b94084f1fb8/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:35.412602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.413426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.413754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.423646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.426459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.439684 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:35.441109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.516210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5067e-ad08-4239-913b-9a31c194d43a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.524242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.525013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.525293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.525566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] PATH_INFO: `/v3/users/bff59be9006440bfb0efb34c45e138c4` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.527259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff59be9006440bfb0efb34c45e138c4)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.528753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.551176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1070/2126] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:35.556030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.557702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.607527 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff59be9006440bfb0efb34c45e138c4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:35.624119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff59be9006440bfb0efb34c45e138c4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:35.636218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:35.636429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74318eb7-851b-49ef-b0b8-81446c4e8036 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.642743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.643517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.643687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.643960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.647865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff59be9006440bfb0efb34c45e138c4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:35.649208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff59be9006440bfb0efb34c45e138c4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:35.655692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.657221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36630a43-6f70-4d4b-a7e5-b131ec2207cc service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.662454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff59be9006440bfb0efb34c45e138c4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:35.667928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1071/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:35.670806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1057/2128] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:35 2022] DELETE /identity/v3/users/bff59be9006440bfb0efb34c45e138c4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:35.675381 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.675666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.675939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.697090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:35.698572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.791246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1072/2129] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:35.823216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:35.826321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:35.950294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68af3a-5e9f-4734-9638-dfea3e29337f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:35.955161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:35.956481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:35.956923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:35.957285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:35.974186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:35.976046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad8c37-6559-4bd8-bc39-282d53d03926 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:35.990264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1058/2130] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:36.222500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:36.226520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.246074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1163d8f-4733-4edc-acbd-7c6ed3ef4c0b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:36.279992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:36.282169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:36.283236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:36.285556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] PATH_INFO: `/v3/users/de1e14243872408fb95d217220f5063e` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:36.288308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1e14243872408fb95d217220f5063e)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:36.292606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:36.305753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:36.389343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1e14243872408fb95d217220f5063e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.410495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1e14243872408fb95d217220f5063e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.430453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1e14243872408fb95d217220f5063e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.431393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1e14243872408fb95d217220f5063e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.447882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1e14243872408fb95d217220f5063e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.463798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:36.464640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:36.464640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:36.471088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1073/2131] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:36 2022] DELETE /identity/v3/users/de1e14243872408fb95d217220f5063e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:36.472923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:36.475432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.494456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:36.506277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.585987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371e7f-ec9f-4386-b461-abd9ca6cfa23 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:36.598685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:36.599268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:36.599569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:36.599807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] PATH_INFO: `/v3/projects/64a19d6ad79f4b6397cbd7e5bad1b75b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:36.600961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a19d6ad79f4b6397cbd7e5bad1b75b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:36.602946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:36.674619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1059/2132] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:36.685944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:36.688762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.721967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a19d6ad79f4b6397cbd7e5bad1b75b', 'request_id': 'req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.738088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a19d6ad79f4b6397cbd7e5bad1b75b', 'request_id': 'req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.756180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a19d6ad79f4b6397cbd7e5bad1b75b', 'request_id': 'req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.783041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] The token cache is being invalidate because project 64a19d6ad79f4b6397cbd7e5bad1b75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:36.783780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.784740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-aa0c8b11-9aa5-4a35-9bad-5741bf1383ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:36.790017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1074/2133] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:36 2022] DELETE /identity/v3/projects/64a19d6ad79f4b6397cbd7e5bad1b75b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:36.790017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:36.790017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:36.790268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:36.821683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215b0b1-b3b4-40ef-b10a-05dc2fbdd255 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:36.840571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] MFA Rules not processed for user `b5b22eed68ce4d879de69b94084f1fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:36.842527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.847493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:36.898654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:36.923465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:36.924315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:36.924796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:36.925607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:36.928053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:36.977550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1075/2134] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:36.979043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:36.993947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:37.037275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:37.038838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:37.051987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1060/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:37.053018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:37.055111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:37.140891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c04a-c5d6-4a85-9d1b-9c5c53fe29ef None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:37.146875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:37.148659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:37.149015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:37.149356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] PATH_INFO: `/v3/users/795ae5ad90a5455cb85dbccd7e184873` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:37.150094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b5301-d5ac-40ca-9b1b-62509dc29ed4 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:37.150862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=795ae5ad90a5455cb85dbccd7e184873)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:37.152982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:37.161839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:37.163300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:37.163660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:37.163993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:37.179943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:37.181654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:37.194765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1061/2136] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:37.217860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795ae5ad90a5455cb85dbccd7e184873'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.228319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795ae5ad90a5455cb85dbccd7e184873'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.244646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795ae5ad90a5455cb85dbccd7e184873'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.245520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795ae5ad90a5455cb85dbccd7e184873'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.255080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795ae5ad90a5455cb85dbccd7e184873'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.262950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1076/2137] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:36 2022] DELETE /identity/v3/users/795ae5ad90a5455cb85dbccd7e184873 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:37.642007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:37.643792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:37.678723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830e90f-2c69-4bba-a450-9109ae7d8015 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:37.768730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:37.769620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:37.769905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:37.770619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] PATH_INFO: `/v3/projects/ba045cda31eb4c0db873a74df91ee00d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:37.772363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba045cda31eb4c0db873a74df91ee00d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:37.775584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:37.793897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:37.796149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:37.819709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcf8e2-5b33-485f-b9c0-155f3b2d98c2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:37.827193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:37.828346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:37.828627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:37.828905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] PATH_INFO: `/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:37.830216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e9ec11146d4be98f7d2ae02b01c97d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:37.833254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:37.865462 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:37.880825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:37.898061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba045cda31eb4c0db873a74df91ee00d', 'request_id': 'req-8db7b8c0-f5b2-4717-b71c-791307ec5389'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.907083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba045cda31eb4c0db873a74df91ee00d', 'request_id': 'req-8db7b8c0-f5b2-4717-b71c-791307ec5389'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.907710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e9ec11146d4be98f7d2ae02b01c97d', 'request_id': 'req-1fec0d61-f376-41d6-93bc-2a0fb3314204'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.916127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba045cda31eb4c0db873a74df91ee00d', 'request_id': 'req-8db7b8c0-f5b2-4717-b71c-791307ec5389'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.918191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e9ec11146d4be98f7d2ae02b01c97d', 'request_id': 'req-1fec0d61-f376-41d6-93bc-2a0fb3314204'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.930466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e9ec11146d4be98f7d2ae02b01c97d', 'request_id': 'req-1fec0d61-f376-41d6-93bc-2a0fb3314204'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.938628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] The token cache is being invalidate because project ba045cda31eb4c0db873a74df91ee00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:37.939058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.939823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.941878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1062/2138] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:37 2022] DELETE /identity/v3/projects/ba045cda31eb4c0db873a74df91ee00d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:37.946814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:37.948494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:37.952227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] The token cache is being invalidate because project e7e9ec11146d4be98f7d2ae02b01c97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:37.953334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.954683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1fec0d61-f376-41d6-93bc-2a0fb3314204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:37.957153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1077/2139] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:37 2022] DELETE /identity/v3/projects/e7e9ec11146d4be98f7d2ae02b01c97d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:37.961975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6c9b-9df5-4195-a5a3-780b2c37ae9d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:37.963155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6c9b-9df5-4195-a5a3-780b2c37ae9d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:37.963648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d6c9b-9df5-4195-a5a3-780b2c37ae9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:37.992942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-9f0d6c9b-9df5-4195-a5a3-780b2c37ae9d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:37.995117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9f0d6c9b-9df5-4195-a5a3-780b2c37ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.035020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7b8c0-f5b2-4717-b71c-791307ec5389 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.096145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1078/2140] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:38.097661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.097999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.098296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.126953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.127733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.128080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.128430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:38.128784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] PATH_INFO: `/v3/users/ff5c1930bb934cb684133e847456adc3` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.130039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.132311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5c1930bb934cb684133e847456adc3)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.135171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.140495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1079/2141] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:38.145276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.156938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e148-36ff-42c4-a884-f11e6278fb01 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.167089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.168003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.168428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.168897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.169949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:38.170463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.173361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.193613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1080/2142] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:59:38.199690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.213173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75c357-213f-4afc-9f45-b6b82d21867a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.220929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.221604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.221856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.222136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.222814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.224597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5c1930bb934cb684133e847456adc3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.224949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.240910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5c1930bb934cb684133e847456adc3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.257197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5c1930bb934cb684133e847456adc3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.257981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5c1930bb934cb684133e847456adc3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.259564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:38.270661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5c1930bb934cb684133e847456adc3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.271582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1081/2143] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:38.272545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.274528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.280722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1063/2144] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:37 2022] DELETE /identity/v3/users/ff5c1930bb934cb684133e847456adc3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:38.389473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fc3a2-8a43-4228-84fd-fb01a6690cca admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.402007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:38.487501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.488147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.488430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.488698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.489351 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.496288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.552066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1082/2145] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:38.556008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.570516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8fcc3-1684-465e-b34b-fc628e0bf65d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.577202 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.577929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.578206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.578512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.579989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.581843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.600678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1064/2146] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:38.605315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.643659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef341279-4b53-4be9-8fd0-c10b106c37af admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.655504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.656207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.656490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.656764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.657855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.659516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53915b7-acbe-409b-9d26-c4b88357abde admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.672771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1083/2147] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:38.676658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.687616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09ffcc-6eed-47a6-affc-adf2b237f55f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.692730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.693622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.694047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.694546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] PATH_INFO: `/v3/projects/cec3b0cd78124db5ab9d78874edbc6e4/users/ae0452f3117248a4b76e3c8938d43e34/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.705549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6070-4fb5-457f-a80f-d099311eb510 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.706693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6070-4fb5-457f-a80f-d099311eb510 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.706693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f6070-4fb5-457f-a80f-d099311eb510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.715879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec3b0cd78124db5ab9d78874edbc6e4, user_id=ae0452f3117248a4b76e3c8938d43e34, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.718171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.730704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-928f6070-4fb5-457f-a80f-d099311eb510 None None] MFA Rules not processed for user `b5b22eed68ce4d879de69b94084f1fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:38.732990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-928f6070-4fb5-457f-a80f-d099311eb510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.739413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1065/2148] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:38 2022] PUT /identity/v3/projects/cec3b0cd78124db5ab9d78874edbc6e4/users/ae0452f3117248a4b76e3c8938d43e34/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:38.748085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.750278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.810004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1084/2149] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:38.811459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.811748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.812159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.841013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf7d35-d602-4dda-aada-778ee5f99d29 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.847006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:38.847670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.847948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.848210 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] PATH_INFO: `/v3/projects/52a4b1078ba84697b286326f3a030401` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.849391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a4b1078ba84697b286326f3a030401)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:38.851341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:38.856820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] MFA Rules not processed for user `ae0452f3117248a4b76e3c8938d43e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:38.858028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.914038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a4b1078ba84697b286326f3a030401', 'request_id': 'req-e2f81955-1ba1-4988-a06a-78bd85675310'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.921147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a4b1078ba84697b286326f3a030401', 'request_id': 'req-e2f81955-1ba1-4988-a06a-78bd85675310'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.930655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a4b1078ba84697b286326f3a030401', 'request_id': 'req-e2f81955-1ba1-4988-a06a-78bd85675310'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.938309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1085/2150] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:38.943288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:38.946415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] The token cache is being invalidate because project 52a4b1078ba84697b286326f3a030401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:38.946721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.947398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e2f81955-1ba1-4988-a06a-78bd85675310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:38.950298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1066/2151] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:38 2022] DELETE /identity/v3/projects/52a4b1078ba84697b286326f3a030401 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:38.955404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56dc3e-1ad3-4789-8561-a9af53ae2503 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:38.956558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7321cdf-7646-4dad-a0be-441d00bf612b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.956850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7321cdf-7646-4dad-a0be-441d00bf612b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.957124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7321cdf-7646-4dad-a0be-441d00bf612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.984671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:38.985574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a7321cdf-7646-4dad-a0be-441d00bf612b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:38.987214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a7321cdf-7646-4dad-a0be-441d00bf612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.027876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.028947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.029647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.030167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.031477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.035790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.036753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:39.070431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1067/2152] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:39.081139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.082204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.086541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1086/2153] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:39.087753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.088427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.090994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.124448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:39.126925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.191383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a336c9-1af2-4068-9838-d9344f71e512 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.198573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.199458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.199887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.200164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.201127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.203650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.219376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1068/2154] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:39.221234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.236256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6675cec-b937-4425-964e-6db831fd3ecd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.245655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1087/2155] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:39.245853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.246273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.246722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.247063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.248164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:39.248164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.249749 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.251514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.270799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1069/2156] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:59:39.271808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.285181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4609b0-d4fc-4c5e-8759-a32cba4df611 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.290878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d46388-57c5-458e-ab35-f2114bfdf28e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.291373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.291928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.292369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.292677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.293526 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.296069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.301257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.301922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.302273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.302613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.303308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.305900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.318423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1088/2157] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:39.321254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.332712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa95ed-5af6-48a1-914d-349d6413d66e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.335690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1070/2158] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:39.338050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.338991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.339364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.339510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.339754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] PATH_INFO: `/v3/projects/f2c6f4c35b2c414ba55792031832c1e0/users/bc3857094a224f73a931114abd371cf0/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.359004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c6f4c35b2c414ba55792031832c1e0, user_id=bc3857094a224f73a931114abd371cf0, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.362269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.378696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1089/2159] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:39 2022] PUT /identity/v3/projects/f2c6f4c35b2c414ba55792031832c1e0/users/bc3857094a224f73a931114abd371cf0/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:39.382823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.422974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5eba6-7cd9-41cf-a200-e9edb87312ff admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.461274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78d88-7297-45f2-8a4b-4c3ecc7b00e7 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.468299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.468975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.469262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.469541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.470423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.472687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.487382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1090/2160] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:39.488157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.489603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:39.495175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:39.499360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d6502-589b-4379-93ff-0a93bd84e421 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.506919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.507777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.508188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.508394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.508805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] PATH_INFO: `/v3/projects/f2c6f4c35b2c414ba55792031832c1e0/users/bc3857094a224f73a931114abd371cf0/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.509087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.509449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.509808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.511518 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c6f4c35b2c414ba55792031832c1e0, user_id=bc3857094a224f73a931114abd371cf0, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.514166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a583-fdac-4e54-90dc-a02eaac5ba9a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.524982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.527837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.536861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1091/2161] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:39 2022] PUT /identity/v3/projects/f2c6f4c35b2c414ba55792031832c1e0/users/bc3857094a224f73a931114abd371cf0/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:39.538618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.539560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.539560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.574714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] MFA Rules not processed for user `bc3857094a224f73a931114abd371cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:39.576291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.589312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1071/2162] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:39.597353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.619057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8679b71-7c4d-47b0-ae12-e3b1553e46bd admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.686293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1092/2163] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:39.692155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.694808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.722980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.723703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.724199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.724549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.726170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.730121 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.750430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1072/2164] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:39.755035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.768009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199db11-6020-4ee7-974b-851776a43d7b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.777205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.778482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.778818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f0f37-796a-482a-8525-cad6926f83c3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.782905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.783541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.784247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.784247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.784786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.785886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.786333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.788503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.799110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.803926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1073/2165] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:39.807841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.808892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.822693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713277-ed0c-4c55-bd0e-c97fbb19605f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.827579 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1093/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:39.830058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.830963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.831397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.831773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] PATH_INFO: `/v3/projects/3f5cee44d2f240c6bd8b5c538fc13956/users/014ea7a1f8124a6b83083e367c0a1db2/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.851536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5cee44d2f240c6bd8b5c538fc13956, user_id=014ea7a1f8124a6b83083e367c0a1db2, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.855445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70310bce-dcc2-4b56-b211-1bda4c12c480 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:39.864199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:39.885846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc140eba-6df0-4d4f-a70f-731e591ce555 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:39.907813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1074/2167] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:39 2022] PUT /identity/v3/projects/3f5cee44d2f240c6bd8b5c538fc13956/users/014ea7a1f8124a6b83083e367c0a1db2/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:39.914792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.915077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.915351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.946867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] MFA Rules not processed for user `014ea7a1f8124a6b83083e367c0a1db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:39.948104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:39.972918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:39.974166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:39.974643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:39.975047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] PATH_INFO: `/v3/users/ae0452f3117248a4b76e3c8938d43e34` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:39.976743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0452f3117248a4b76e3c8938d43e34)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:39.979915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:40.008927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:40.037400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1075/2168] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:40.044086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0452f3117248a4b76e3c8938d43e34'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.050966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:40.052467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:40.053537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0452f3117248a4b76e3c8938d43e34'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.068650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0452f3117248a4b76e3c8938d43e34'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.069463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0452f3117248a4b76e3c8938d43e34'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.077091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3e154644-9ed5-44d6-b141-f035a992b2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0452f3117248a4b76e3c8938d43e34'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.083592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1094/2169] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:39 2022] DELETE /identity/v3/users/ae0452f3117248a4b76e3c8938d43e34 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:40.087915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:40.088213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:40.088472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:40.104897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:40.108630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:40.109919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:40.144320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0ccfa-11f6-4450-b889-9bd2bd09716a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:40.188441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1095/2170] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:40.195086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:40.197091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:40.221374 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:40.221999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:40.222278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:40.222566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:40.224890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:40.264048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155e523-e7df-436a-b2cf-2edf454bf508 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:40.268737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:40.269537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:40.269819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:40.270092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:40.280011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:40.281581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:40.292161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1096/2171] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:40.295634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:40.297359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:40.312227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1076/2172] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:40.749577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:40.751067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:40.774965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6506a-0a28-4feb-962b-e0acc4ac472a service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:40.783830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:40.784754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:40.785370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:40.785703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] PATH_INFO: `/v3/projects/cec3b0cd78124db5ab9d78874edbc6e4` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:40.787286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec3b0cd78124db5ab9d78874edbc6e4)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:40.789556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:40.865679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:40.903041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec3b0cd78124db5ab9d78874edbc6e4', 'request_id': 'req-98e79fa7-1628-4b50-a838-68a4f0748ee9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.915363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec3b0cd78124db5ab9d78874edbc6e4', 'request_id': 'req-98e79fa7-1628-4b50-a838-68a4f0748ee9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.928254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec3b0cd78124db5ab9d78874edbc6e4', 'request_id': 'req-98e79fa7-1628-4b50-a838-68a4f0748ee9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.949586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] The token cache is being invalidate because project cec3b0cd78124db5ab9d78874edbc6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:40.950030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.950906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:40.954449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1097/2173] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:40 2022] DELETE /identity/v3/projects/cec3b0cd78124db5ab9d78874edbc6e4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:40.958914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:40.970846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e467c-f0b3-4a9a-88c4-b194d65fc831 service glance] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:41.039344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:41.046715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:41.046715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:41.046715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] PATH_INFO: `/v3/users/bc3857094a224f73a931114abd371cf0` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:41.047789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3857094a224f73a931114abd371cf0)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:41.050711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:41.097081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:41.117023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3857094a224f73a931114abd371cf0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.127041 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3857094a224f73a931114abd371cf0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.143722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3857094a224f73a931114abd371cf0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.144491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3857094a224f73a931114abd371cf0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.152611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-22be43af-17b4-4f5e-8b5d-b525cafdd629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3857094a224f73a931114abd371cf0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.159339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1077/2174] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:40 2022] DELETE /identity/v3/users/bc3857094a224f73a931114abd371cf0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:41.516186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:41.517882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:41.551157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e79fa7-1628-4b50-a838-68a4f0748ee9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:41.661778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:41.662507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:41.662961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:41.663275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] PATH_INFO: `/v3/projects/f2c6f4c35b2c414ba55792031832c1e0` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:41.665675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c6f4c35b2c414ba55792031832c1e0)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:41.669937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:41.755327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c6f4c35b2c414ba55792031832c1e0', 'request_id': 'req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.799047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c6f4c35b2c414ba55792031832c1e0', 'request_id': 'req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.814702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c6f4c35b2c414ba55792031832c1e0', 'request_id': 'req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.848771 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] The token cache is being invalidate because project f2c6f4c35b2c414ba55792031832c1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:41.849072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.853160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-bcd2cd49-fa17-4fff-ab26-a39a34956c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:41.860452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1098/2175] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:41 2022] DELETE /identity/v3/projects/f2c6f4c35b2c414ba55792031832c1e0 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:59:41.866196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:41.868222 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:41.868590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:41.898619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:41.899862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:42.011739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1078/2176] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:42.025898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.026206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.026586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.048313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:42.049584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:42.058901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1099/2177] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:42.063589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.074247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f9f9-a79a-49de-9b0c-74508dc24815 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.079618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.080330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.080522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.080816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.081475 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:42.081770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.084357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.096173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1079/2178] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:42.100636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.111252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f1a2e-8c24-4110-b104-a9761e2ff7b8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.118455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.119224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.119504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.119763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.121588 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.122419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.158573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1100/2179] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:42.162918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.175238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804c33e-cad4-46bd-9307-dc47d1e6c665 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.262054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.262746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.263020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.263289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.263985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.266074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.291620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1080/2180] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:42.295972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.307240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2661392-f7ba-4a69-a3f8-caa36d02310d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.313454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.314309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.314756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.315152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.316266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.318100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.329945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1101/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:42.334278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.345510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69873a74-ff67-4556-a073-d4ff46eabcdb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.350296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.350961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.351237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.351520 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.352319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.353950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6cf32-9926-40ef-9d3d-d534c449dcd3 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.368901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1081/2182] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:42.371056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.382043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbbecd-c07a-4253-b944-50fec3db0b21 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.387313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.388034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.388310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.388576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] PATH_INFO: `/v3/projects/ceeef530483f4b9081d8ae6f4baa1665/users/4901345656d54e8a95dbc94a4ff95317/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.402120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:42.406941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceeef530483f4b9081d8ae6f4baa1665, user_id=4901345656d54e8a95dbc94a4ff95317, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.410119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.426582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1102/2183] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:42 2022] PUT /identity/v3/projects/ceeef530483f4b9081d8ae6f4baa1665/users/4901345656d54e8a95dbc94a4ff95317/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:42.437489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.438013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.438013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.466556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:42.471435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] MFA Rules not processed for user `4901345656d54e8a95dbc94a4ff95317`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:42.473486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:42.548866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1082/2184] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:42.553309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.627896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f02878-f4c6-4181-aa08-52c4d5081b5a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.637098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.637748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.638037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.638308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.638996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.641034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.677440 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1103/2185] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:42.678791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.690646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d9da-b733-4af3-8fe6-9cb02a526757 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.752066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.752694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.753029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.753309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.753971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.756094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.785487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1083/2186] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:42 2022] POST /identity/v3/users => generated 310 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:42.787001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.798503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc2ce5-0815-4b91-acd7-8b1da912b258 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.804473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.805029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.805385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.805546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.806205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.807898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.821703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1104/2187] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:42.826523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.837315 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe09b-4488-4871-b79c-ea1befaccb7a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.843603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.844421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.844673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.844945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] PATH_INFO: `/v3/projects/2e3511469e6447bd9b45580947a67d77/users/3895ff6453e0409e9af7d5e921ff50df/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.866709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3511469e6447bd9b45580947a67d77, user_id=3895ff6453e0409e9af7d5e921ff50df, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.869409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.884987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1084/2188] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:42 2022] PUT /identity/v3/projects/2e3511469e6447bd9b45580947a67d77/users/3895ff6453e0409e9af7d5e921ff50df/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:42.889184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:42.903212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ebfab-185c-48ce-bf84-43d043dee568 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:42.971061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:42.971700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:42.971975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:42.972396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:42.973033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:42.974810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ff77-dd12-4285-9bd6-525cd0766585 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:42.991639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1105/2189] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:42.996405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:43.008688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e299e-75a5-4e2b-b043-3e64ac950329 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:43.014723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:43.015698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:43.016047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:43.016331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] PATH_INFO: `/v3/projects/2e3511469e6447bd9b45580947a67d77/users/3895ff6453e0409e9af7d5e921ff50df/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:43.018401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3511469e6447bd9b45580947a67d77, user_id=3895ff6453e0409e9af7d5e921ff50df, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:43.022018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:43.042843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1085/2190] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:42 2022] PUT /identity/v3/projects/2e3511469e6447bd9b45580947a67d77/users/3895ff6453e0409e9af7d5e921ff50df/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:43.051212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:43.052358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:43.052495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:43.083812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] MFA Rules not processed for user `3895ff6453e0409e9af7d5e921ff50df`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:43.084874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:43.147130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:43.170692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1106/2191] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:43 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:43.181982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:43.183805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:43.242653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e9b0-d3ea-4275-997d-d4bacbf7733c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:43.247754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:43.248765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:43.249395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:43.249892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:43.267010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:43.268704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:43.273787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:43.290113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1086/2192] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:43 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:43.513775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:43.533600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888a478-3273-485a-b080-d2e48c997d19 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:43.537560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:43.538207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:43.538529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:43.538803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:43.636565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:43.638034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:43.648205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1107/2193] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:43 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:46.789376 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:46.791459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:46.874800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfa49-95df-47ea-b646-f3cf723226b2 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:46.880436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:46.881033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:46.881348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:46.881647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] PATH_INFO: `/v3/users/bd55568bf88346b794922ef8555f9654` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:46.882968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd55568bf88346b794922ef8555f9654)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:46.885029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:46.942069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd55568bf88346b794922ef8555f9654'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:46.951150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd55568bf88346b794922ef8555f9654'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:46.966405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd55568bf88346b794922ef8555f9654'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:46.967276 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd55568bf88346b794922ef8555f9654'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:46.979150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1ff8e28-f632-450e-b93f-20c2ac89a764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd55568bf88346b794922ef8555f9654'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:46.990648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1087/2194] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:46 2022] DELETE /identity/v3/users/bd55568bf88346b794922ef8555f9654 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:47.291483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.304305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd006b4-ed92-4bd8-ad72-b4d6643689cf service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.409636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.410317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.410653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.410973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] PATH_INFO: `/v3/projects/8ac3cf4af51e419c848daba7adc86024` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.412897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac3cf4af51e419c848daba7adc86024)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.415306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.475159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac3cf4af51e419c848daba7adc86024', 'request_id': 'req-46b7610b-adbb-4adc-addb-df5e59285365'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:47.480871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac3cf4af51e419c848daba7adc86024', 'request_id': 'req-46b7610b-adbb-4adc-addb-df5e59285365'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:47.488330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac3cf4af51e419c848daba7adc86024', 'request_id': 'req-46b7610b-adbb-4adc-addb-df5e59285365'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:47.513636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] The token cache is being invalidate because project 8ac3cf4af51e419c848daba7adc86024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:47.514294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:47.515074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-46b7610b-adbb-4adc-addb-df5e59285365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:47.517245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1108/2195] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:47 2022] DELETE /identity/v3/projects/8ac3cf4af51e419c848daba7adc86024 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:47.521938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.522218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.528972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.557816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:47.559303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:47.648365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1088/2196] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:47.656231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.656734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.658115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.679775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:47.680991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:47.689327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1109/2197] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:47.693786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.704109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a29d3-2215-4afb-97d1-72faff136ee9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.708734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.709353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.709621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.709887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.710579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:47.710882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.712929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.730263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1089/2198] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:59:47.731557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.742086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fad7bd-c93f-4088-ba55-5aacc756df3a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.747297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.747928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.748205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.748474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.749215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.751267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.758044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:47.783531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1110/2199] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:47 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:47.787765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.794335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:47.798586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7be34-72a6-4903-b020-2e515d6f8fce admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.870923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.871601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.871965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.872300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.872964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.875067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.903980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1090/2200] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:47 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:47.909401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.923017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a335180-bed5-4676-b457-d662813a3d3a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.937237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.937889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.938177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.938484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.939148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.940842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.954984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1111/2201] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:47.959740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:47.971420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f83a5-483c-4a09-aa8a-60adf824c728 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:47.977289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:47.977923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:47.978214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:47.978533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:47.979189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:47.980824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52f48-2489-4f58-9caa-ce500fd1e3a4 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:47.992764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1091/2202] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:47.997144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:48.008481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72604a6c-09b8-4577-8a2d-a68ff0bb2175 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:48.013505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:48.014174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:48.014488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:48.014804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] PATH_INFO: `/v3/projects/b3fa762826924accaa8012d7ceaf17eb/users/87c4f9c4b9334b66ac89e01a278436bc/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:48.032951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fa762826924accaa8012d7ceaf17eb, user_id=87c4f9c4b9334b66ac89e01a278436bc, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:48.035706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:48.050349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1112/2203] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:47 2022] PUT /identity/v3/projects/b3fa762826924accaa8012d7ceaf17eb/users/87c4f9c4b9334b66ac89e01a278436bc/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:48.056718 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:48.057168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:48.057616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:48.079729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:48.081058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:48.145607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1092/2204] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:48.156263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:48.159262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:48.230357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055788a6-e966-4e51-bcd5-2a9a44ee529e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:48.236558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:48.237180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:48.237463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:48.237734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:48.247610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:48.249209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8e45-0832-46bc-a5d8-0b17b4da4234 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:48.263571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1113/2205] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:54.030424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:54.032768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:54.073139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a885c-8fb6-4343-900a-2b4dffe17940 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:54.093916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:54.095096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:54.096049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:54.096528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] PATH_INFO: `/v3/users/4901345656d54e8a95dbc94a4ff95317` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:54.102870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4901345656d54e8a95dbc94a4ff95317)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:54.106348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:54.219127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4901345656d54e8a95dbc94a4ff95317'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:54.239738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4901345656d54e8a95dbc94a4ff95317'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:54.296145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4901345656d54e8a95dbc94a4ff95317'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:54.297914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4901345656d54e8a95dbc94a4ff95317'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:54.309629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:54.318139 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4901345656d54e8a95dbc94a4ff95317'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:54.332703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1093/2206] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:54 2022] DELETE /identity/v3/users/4901345656d54e8a95dbc94a4ff95317 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:54.337494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:54.338107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:54.338107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:54.376758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:54.379468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:54.523249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1114/2207] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:54.535963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:54.537416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:54.631050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c0713-fa33-4156-a89e-6d5e20d89859 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:54.636760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:54.637883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:54.638332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:54.639135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:54.651542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:54.653695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:54.675891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1094/2208] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:55.185683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:55.187370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:55.199722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5c0c0-d5a2-4e7e-ad11-c5e4a8e61392 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:55.204538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:55.205165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:55.205448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:55.205719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] PATH_INFO: `/v3/projects/ceeef530483f4b9081d8ae6f4baa1665` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:55.207043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceeef530483f4b9081d8ae6f4baa1665)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:55.208940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:55.281574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:55.284399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceeef530483f4b9081d8ae6f4baa1665', 'request_id': 'req-850b99c2-19aa-4eea-b5fc-8f51201369e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.294353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceeef530483f4b9081d8ae6f4baa1665', 'request_id': 'req-850b99c2-19aa-4eea-b5fc-8f51201369e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.308632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceeef530483f4b9081d8ae6f4baa1665', 'request_id': 'req-850b99c2-19aa-4eea-b5fc-8f51201369e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.330764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] The token cache is being invalidate because project ceeef530483f4b9081d8ae6f4baa1665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:55.331778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.331963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-850b99c2-19aa-4eea-b5fc-8f51201369e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.333710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1115/2209] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:55 2022] DELETE /identity/v3/projects/ceeef530483f4b9081d8ae6f4baa1665 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:55.338493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:55.351123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb82f9a-7880-4875-be78-78cada9b133e service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:55.442161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:55.442879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:55.443179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:55.443532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] PATH_INFO: `/v3/users/3895ff6453e0409e9af7d5e921ff50df` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:55.445459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3895ff6453e0409e9af7d5e921ff50df)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:55.448052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:55.528349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3895ff6453e0409e9af7d5e921ff50df'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.542250 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3895ff6453e0409e9af7d5e921ff50df'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.563914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3895ff6453e0409e9af7d5e921ff50df'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.564891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3895ff6453e0409e9af7d5e921ff50df'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.597274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3895ff6453e0409e9af7d5e921ff50df'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:55.617397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1095/2210] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 19:59:55 2022] DELETE /identity/v3/users/3895ff6453e0409e9af7d5e921ff50df => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:55.938262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:55.938829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:55.939027 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.013534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] MFA Rules not processed for user `87c4f9c4b9334b66ac89e01a278436bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:56.016404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:56.137356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1116/2211] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:56.151739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.153802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:56.179186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:56.271388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f556a24-bbf5-4aeb-b31a-c753cd7b221b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.283523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.284095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.284217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.284673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.284914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.286648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:56.309162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.310936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613e3ee-eec6-4a93-b718-0899a36416c1 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.321671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1096/2212] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:56.388415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb94ef5-1719-4ff9-b456-79cc809a90c5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.401724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.402749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.402749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.402963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] PATH_INFO: `/v3/projects/2e3511469e6447bd9b45580947a67d77` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.404058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3511469e6447bd9b45580947a67d77)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.406778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.479198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3511469e6447bd9b45580947a67d77', 'request_id': 'req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:56.486133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3511469e6447bd9b45580947a67d77', 'request_id': 'req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:56.493404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3511469e6447bd9b45580947a67d77', 'request_id': 'req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:56.497722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:56.510312 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] The token cache is being invalidate because project 2e3511469e6447bd9b45580947a67d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:59:56.510657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:56.512219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e383c840-f2f2-4a9b-a249-0af0e10b9e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:59:56.514240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1117/2213] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 19:59:56 2022] DELETE /identity/v3/projects/2e3511469e6447bd9b45580947a67d77 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:56.519339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.519857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.519857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.547513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:56.549449 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:56.628174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1097/2214] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:56.637830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.638119 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.638416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.658978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:56.660204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:56.668406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1118/2215] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:56.672828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.682937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e7298-2f66-4a53-a770-b68889a5b51f None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.688784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.689487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.689836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.690176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.691070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:59:56.691445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.693316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.706652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1098/2216] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 19:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:59:56.711818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.723618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53661bea-bf64-4b97-a372-eb4312b65e1b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.730317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.730992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.731774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.732062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.732864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.735258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.782231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1119/2217] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:56 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:56.786218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.797181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eab67-6b6c-478a-959c-48a68ddc0fff admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.892899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.894040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.895088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.895742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.897425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.899870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.932137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1099/2218] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:56 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:56.936065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.947613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59714a2a-65af-4d17-b1d8-9916db42c789 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.953691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.954358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.954640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.954910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.955560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.957182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.973238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1120/2219] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:56.974257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.991929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f577d4-cb24-48a4-9a28-6849654937b9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.003736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.004384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.004669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.004953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.005663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.007679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c208a1-2b17-419d-857e-c4983a11c49e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.023214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1100/2220] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:59:57.033389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.046503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a4a7-cb02-4ce8-9154-bb0c148d6e0b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.051357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.052421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.052421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.052618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] PATH_INFO: `/v3/projects/d45c9926778149debeb58d9f25d8aa5d/users/0506be82c13548d2a4bdfc4e5f1af36d/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.070963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d45c9926778149debeb58d9f25d8aa5d, user_id=0506be82c13548d2a4bdfc4e5f1af36d, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.073805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.094429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1121/2221] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:57 2022] PUT /identity/v3/projects/d45c9926778149debeb58d9f25d8aa5d/users/0506be82c13548d2a4bdfc4e5f1af36d/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:57.108528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.108935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.109318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.137711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:57.149832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] MFA Rules not processed for user `0506be82c13548d2a4bdfc4e5f1af36d`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:57.151722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:57.246784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1101/2222] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:57.251396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.361441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1e8a4-6129-4d44-9200-4452d9fd08e0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.368053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.368891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.369306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.369849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.370496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.375293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.415930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1122/2223] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:57 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:57.419737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.434103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f62a-bfcf-459e-9c96-071565cde773 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.532487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.533987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.534525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.535027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.535910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.539563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.582023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1102/2224] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:57 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:57.585580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.599138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069f299-49ec-4564-80b7-37ff182317ff admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.605257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.605906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.606763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.607103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.607825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.610143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.627254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1123/2225] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:57.634900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.653283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e79cb-163a-4942-bfa4-5ae46796e8e0 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.661569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.662595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.662971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.663316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.664113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.666090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9506f-72a5-4ffb-b97f-a6afa1be8339 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.696539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1103/2226] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:57.701765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:57.720451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404b6af-1905-4a87-b297-53a5648d97c0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:57.770795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:57.771504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.771801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.772066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] PATH_INFO: `/v3/projects/7d82e471344d488f8df052152fb807b2/users/e5d9755d2c0f45f395397a2dc1539fec/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.794683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d82e471344d488f8df052152fb807b2, user_id=e5d9755d2c0f45f395397a2dc1539fec, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:57.798788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:57.845450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1124/2227] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:57 2022] PUT /identity/v3/projects/7d82e471344d488f8df052152fb807b2/users/e5d9755d2c0f45f395397a2dc1539fec/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:57.847272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8aa-f1b1-4392-b4a2-e88bf66ac6d8 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:57.847554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8aa-f1b1-4392-b4a2-e88bf66ac6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:57.847845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8aa-f1b1-4392-b4a2-e88bf66ac6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:57.895172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4cdbe8aa-f1b1-4392-b4a2-e88bf66ac6d8 None None] MFA Rules not processed for user `e5d9755d2c0f45f395397a2dc1539fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:57.896472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4cdbe8aa-f1b1-4392-b4a2-e88bf66ac6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:57.995153 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1104/2228] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:58.001289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.069590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:58.100437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d24c53-6192-4c7d-8e3f-2c7eb4f26ab1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.105910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.106848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.107318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.107804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.108937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.111076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.145145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.145726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.145876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.148639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1125/2229] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 19:59:57 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:59:58.149639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.160392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:58.167666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] MFA Rules not processed for user `87c4f9c4b9334b66ac89e01a278436bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:58.169010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:58.238630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1105/2230] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:58.238871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19135404-7902-4dd1-99df-86b9250ae371 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.244299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.244929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.245356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.245793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.246472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.248489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.278233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1126/2231] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 19:59:58 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:59:58.279404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.291318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae9662-fc9e-4083-865e-ea96ac524e94 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.296301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.296813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.297098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.297367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.298024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.299699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.314123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1106/2232] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:58.320702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.332231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6a17-510c-481f-9b8c-770d26d64c2d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.341509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.345146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.345146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.345146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] PATH_INFO: `/v3/projects/9ae97530f84542f49cb677cbfd47bf03/users/80020e3533c24478891af4bb2c4dbc14/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.366654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae97530f84542f49cb677cbfd47bf03, user_id=80020e3533c24478891af4bb2c4dbc14, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.370014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.391472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.392137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1127/2233] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:58 2022] PUT /identity/v3/projects/9ae97530f84542f49cb677cbfd47bf03/users/80020e3533c24478891af4bb2c4dbc14/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:58.403340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbae2aa-7f59-4300-8014-7bbbb5dd57a7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.470311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.471028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.471361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.471715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.472432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.474144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4101-0678-4df3-b295-bcf59af95d80 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.485827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1107/2234] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 19:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:59:58.490070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.500585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb53fe6-a9dd-4447-803f-1a6712ca2133 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.505259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.505940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.506210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.506520 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] PATH_INFO: `/v3/projects/9ae97530f84542f49cb677cbfd47bf03/users/80020e3533c24478891af4bb2c4dbc14/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.508409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae97530f84542f49cb677cbfd47bf03, user_id=80020e3533c24478891af4bb2c4dbc14, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.511101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.521938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1128/2235] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 19:59:58 2022] PUT /identity/v3/projects/9ae97530f84542f49cb677cbfd47bf03/users/80020e3533c24478891af4bb2c4dbc14/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:59:58.532234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.532515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.533062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.571752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] MFA Rules not processed for user `80020e3533c24478891af4bb2c4dbc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:58.572655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:58.666474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1108/2236] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 19:59:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:58.677585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:58.679656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:58.744580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb02238-7a2e-40b7-b3e2-06fbd41d2dbb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:58.748300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:58.748913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:58.749189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:58.749456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:58.784700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:58.816127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:58.817993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:58.829536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1129/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:01.434406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:01.436212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:01.451280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99281d-fffb-41bf-8e89-a5e4cedc5c12 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:01.455957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:01.456880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:01.457320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:01.457679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:01.481246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:01.523270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:01.524653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:01.534697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1109/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:02.228989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:02.241469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3ce2b-0b08-4328-a327-803bc51aa957 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:02.245616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:02.246253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:02.246927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:02.246927 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:02.256966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:02.258469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4c0c-e172-473d-867d-8956e9eb2c8f service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:02.277261 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1130/2239] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:04.111707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:04.113336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:04.198392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20867a-454f-4975-a1a1-691b43f0bc4d service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:04.203304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:04.203998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:04.204273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:04.204544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] PATH_INFO: `/v3/users/0f08f07de04f4abb9f51d178b0f95bb7` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:04.205697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f08f07de04f4abb9f51d178b0f95bb7)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:04.207783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:04.260762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f08f07de04f4abb9f51d178b0f95bb7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.269569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f08f07de04f4abb9f51d178b0f95bb7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.283072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f08f07de04f4abb9f51d178b0f95bb7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.283770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f08f07de04f4abb9f51d178b0f95bb7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.290838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f08f07de04f4abb9f51d178b0f95bb7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.297578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1110/2240] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:04 2022] DELETE /identity/v3/users/0f08f07de04f4abb9f51d178b0f95bb7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:04.302786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:04.303345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:04.303500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:04.336955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:04.338242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:04.404828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1131/2241] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:04.414747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:04.416203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:04.459775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:04.475891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33308a45-263e-4166-9c14-6a7c1628a47a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:04.479870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:04.480492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:04.480776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:04.481049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:04.490877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:04.492321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:04.503679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1111/2242] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:04.800237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:04.801804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:04.824474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bad4d0-9cde-4994-a65d-78f26b48f016 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:04.830798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:04.831592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:04.832062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:04.832409 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] PATH_INFO: `/v3/projects/d56091e6d4da453fbeea125f6659f11d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:04.833722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56091e6d4da453fbeea125f6659f11d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:04.835693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:04.892576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56091e6d4da453fbeea125f6659f11d', 'request_id': 'req-fe3519db-8eb2-49d5-985d-ae0a514a05cf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.898626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56091e6d4da453fbeea125f6659f11d', 'request_id': 'req-fe3519db-8eb2-49d5-985d-ae0a514a05cf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.907037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56091e6d4da453fbeea125f6659f11d', 'request_id': 'req-fe3519db-8eb2-49d5-985d-ae0a514a05cf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.922161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] The token cache is being invalidate because project d56091e6d4da453fbeea125f6659f11d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:04.922740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.923745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:04.925538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1132/2243] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:04 2022] DELETE /identity/v3/projects/d56091e6d4da453fbeea125f6659f11d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:04.929950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:04.943096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8f17-088e-49d6-8d67-6a391b4861d7 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:05.003249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:05.003932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.004211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.004488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] PATH_INFO: `/v3/users/61870972789541e1a37ac0fb40624e70` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.005646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61870972789541e1a37ac0fb40624e70)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:05.007683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:05.065066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61870972789541e1a37ac0fb40624e70'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.072739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61870972789541e1a37ac0fb40624e70'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.091378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61870972789541e1a37ac0fb40624e70'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.092043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61870972789541e1a37ac0fb40624e70'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.104196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-bd22827c-c855-4f03-9dbc-393097032ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61870972789541e1a37ac0fb40624e70'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.112607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1112/2244] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:04 2022] DELETE /identity/v3/users/61870972789541e1a37ac0fb40624e70 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:05.407672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:05.409298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:05.422537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3519db-8eb2-49d5-985d-ae0a514a05cf admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:05.482002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:05.499953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:05.500646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.500923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.501192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] PATH_INFO: `/v3/projects/c2644b24b54243f2a5a53e1b25b74616` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.502327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2644b24b54243f2a5a53e1b25b74616)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:05.504310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:05.560188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2644b24b54243f2a5a53e1b25b74616', 'request_id': 'req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.572251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2644b24b54243f2a5a53e1b25b74616', 'request_id': 'req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.583041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2644b24b54243f2a5a53e1b25b74616', 'request_id': 'req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.602002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] The token cache is being invalidate because project c2644b24b54243f2a5a53e1b25b74616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:05.602597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.603338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-82e9c06a-60ab-4ae5-9f2a-0830f40f1078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:05.604952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1133/2245] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:05 2022] DELETE /identity/v3/projects/c2644b24b54243f2a5a53e1b25b74616 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:05.610068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.610357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.610700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.630574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:05.632044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:05.696063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1113/2246] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:05.706596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.707098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.707384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.731559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:05.732566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:05.741858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1134/2247] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:05.746636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:05.752976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:05.759787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54b1c7-dea9-4895-8e0d-f8d3ebee4b34 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:05.767660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:05.770162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.770714 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.770993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.771874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:05.772725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:05.775307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:05.794436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1114/2248] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:00:05.796277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:05.807107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f981d-d75a-40d2-af7a-d26641eed214 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:05.818217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:05.819352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.819792 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.820060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:05.820734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:05.822917 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:05.861508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1135/2249] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:05 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:05.865756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:05.876972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e86c0-333d-410b-9264-081469dc414a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:05.997400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:05.998311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:05.998646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:05.998924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.000179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.002826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.035708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1115/2250] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:05 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:06.037334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.048413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6deca-22db-4d28-9b49-aa4887bfba67 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.053299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.053938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.054214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.054519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.055171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.057586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.068788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1136/2251] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:06.072787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.084541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9715-d884-4c43-aac8-4ffd2f1e569c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.090110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.090783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.091065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.091340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.091997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.093664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bcc14-627d-4e44-b8a9-a062e66f7d0a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.106694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1116/2252] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:06.112467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.126796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a98d39-9ea5-4e5b-90fb-cae1552359ab admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.133863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.133863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.133863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.133863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] PATH_INFO: `/v3/projects/828be2d6209d4d7d9d2b5b5822483bed/users/052daf02c895403780682b87894dc723/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.155048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=828be2d6209d4d7d9d2b5b5822483bed, user_id=052daf02c895403780682b87894dc723, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.158227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.176924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1137/2253] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:06 2022] PUT /identity/v3/projects/828be2d6209d4d7d9d2b5b5822483bed/users/052daf02c895403780682b87894dc723/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:06.188192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.188756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.188756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.225163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] MFA Rules not processed for user `052daf02c895403780682b87894dc723`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:06.226866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:06.305806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1117/2254] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:06.310551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.381513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.382969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:06.392358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53d8ab-51b3-4679-83b0-fda18f94ac51 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.397514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.398141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.398467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.398696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b4014-88c6-4951-b1ea-635896a0a054 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.398820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.399412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.401604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.403790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.404553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.404827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.405098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] PATH_INFO: `/v3/users/0506be82c13548d2a4bdfc4e5f1af36d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.406224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0506be82c13548d2a4bdfc4e5f1af36d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.408807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.419423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:06.443899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1138/2255] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:06 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:06.446838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.482556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0506be82c13548d2a4bdfc4e5f1af36d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:06.488580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:06.495853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0506be82c13548d2a4bdfc4e5f1af36d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:06.515756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0506be82c13548d2a4bdfc4e5f1af36d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:06.518495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0506be82c13548d2a4bdfc4e5f1af36d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:06.540804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c767a0-d4ce-419d-9469-720fbfddeb89 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.541001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d5d35010-fc69-4576-8f8b-3bce6c7fdc60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0506be82c13548d2a4bdfc4e5f1af36d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:06.547663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.548317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.548597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.548906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.549561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.551682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.566289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1118/2256] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:06 2022] DELETE /identity/v3/users/0506be82c13548d2a4bdfc4e5f1af36d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:06.571061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.571508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.571918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.585587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1139/2257] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:06 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:06.587688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.590237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:06.599344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:06.600560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:06.608388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8491a-9084-4580-a9ad-6681995ca070 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.684555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1119/2258] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:06.696583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.698327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:06.708510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.709437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.709828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.710219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.711528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.713321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.730993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1140/2259] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:06.735347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.746930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5103a298-c2e1-4ae2-a4e9-710ef633d387 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.752912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.753843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.754187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.754574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] PATH_INFO: `/v3/projects/7a4d4ba81e704583967ee816641c8b07/users/64ccda3e2ce94ae5b8c904bb6dbd29a2/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.775929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4d4ba81e704583967ee816641c8b07, user_id=64ccda3e2ce94ae5b8c904bb6dbd29a2, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.779350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.784167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffad16-c5a4-4426-a273-cdcfde259759 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.789152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.790423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.790774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.791053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.802821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.804540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.806052 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1141/2260] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:06 2022] PUT /identity/v3/projects/7a4d4ba81e704583967ee816641c8b07/users/64ccda3e2ce94ae5b8c904bb6dbd29a2/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:06.811326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.831367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1120/2261] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:00:06.906956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:06.913614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19bddb-5f72-429a-9deb-3e62b0b86014 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.920890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.921700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.922218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.922728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.923549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.925664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d548a86-c46c-457f-8599-8b88c5502135 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.939212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1142/2262] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:06.943602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:06.955614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8eb298-0827-4180-889c-2962b4c2c3a5 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:06.961159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:06.961957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.962348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.963034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] PATH_INFO: `/v3/projects/7a4d4ba81e704583967ee816641c8b07/users/64ccda3e2ce94ae5b8c904bb6dbd29a2/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:06.965529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4d4ba81e704583967ee816641c8b07, user_id=64ccda3e2ce94ae5b8c904bb6dbd29a2, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:06.968517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:06.982212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1121/2263] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:06 2022] PUT /identity/v3/projects/7a4d4ba81e704583967ee816641c8b07/users/64ccda3e2ce94ae5b8c904bb6dbd29a2/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:06.993191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:06.993533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:06.993765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:07.028754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] MFA Rules not processed for user `64ccda3e2ce94ae5b8c904bb6dbd29a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:07.030310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:07.116713 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1143/2264] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:07.132801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:07.134801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:07.211835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21355d92-0e90-4cf9-9fec-33fc6dbc33bb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:07.220875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:07.221581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:07.221929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:07.222246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:07.236831 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:07.239602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:07.251945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1122/2265] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:07.270893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:07.272954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:07.377362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1d36f-57a6-4c13-9c61-c8ccccf1462f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:07.383995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:07.384799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:07.385178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:07.385546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] PATH_INFO: `/v3/projects/d45c9926778149debeb58d9f25d8aa5d` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:07.386883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d45c9926778149debeb58d9f25d8aa5d)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:07.389304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:07.454487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd45c9926778149debeb58d9f25d8aa5d', 'request_id': 'req-ccd28842-3bec-4084-82eb-1a9e9209750a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.462710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd45c9926778149debeb58d9f25d8aa5d', 'request_id': 'req-ccd28842-3bec-4084-82eb-1a9e9209750a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.467944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd45c9926778149debeb58d9f25d8aa5d', 'request_id': 'req-ccd28842-3bec-4084-82eb-1a9e9209750a'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.483933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] The token cache is being invalidate because project d45c9926778149debeb58d9f25d8aa5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:07.484247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.486481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.487919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1144/2266] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:07 2022] DELETE /identity/v3/projects/d45c9926778149debeb58d9f25d8aa5d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:07.492689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:07.499316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:07.508661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba9a6-6f0e-42e7-931a-e18a289b7820 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:07.608332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:07.610928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:07.611306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:07.611788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] PATH_INFO: `/v3/users/e5d9755d2c0f45f395397a2dc1539fec` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:07.613677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d9755d2c0f45f395397a2dc1539fec)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:07.617282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:07.700501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d9755d2c0f45f395397a2dc1539fec'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.711674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d9755d2c0f45f395397a2dc1539fec'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.736168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d9755d2c0f45f395397a2dc1539fec'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.737298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d9755d2c0f45f395397a2dc1539fec'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.746669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d9755d2c0f45f395397a2dc1539fec'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:07.765459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1123/2267] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:07 2022] DELETE /identity/v3/users/e5d9755d2c0f45f395397a2dc1539fec => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:08.147566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:08.151586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:08.168256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28842-3bec-4084-82eb-1a9e9209750a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:08.174902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:08.274162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:08.275075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:08.275450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:08.275928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] PATH_INFO: `/v3/projects/7d82e471344d488f8df052152fb807b2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:08.277391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d82e471344d488f8df052152fb807b2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:08.283177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:08.365286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d82e471344d488f8df052152fb807b2', 'request_id': 'req-68f88c3b-78b3-460e-8758-082fe3b8f920'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.372814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d82e471344d488f8df052152fb807b2', 'request_id': 'req-68f88c3b-78b3-460e-8758-082fe3b8f920'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.388143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d82e471344d488f8df052152fb807b2', 'request_id': 'req-68f88c3b-78b3-460e-8758-082fe3b8f920'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.405269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] The token cache is being invalidate because project 7d82e471344d488f8df052152fb807b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:08.405682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.407139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.410931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1145/2268] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:08 2022] DELETE /identity/v3/projects/7d82e471344d488f8df052152fb807b2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:08.415143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:08.416663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:08.429940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626642ef-b55a-406e-b4c5-2d785b177684 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:08.511462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:08.512493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:08.512817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:08.513099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] PATH_INFO: `/v3/users/80020e3533c24478891af4bb2c4dbc14` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:08.514751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80020e3533c24478891af4bb2c4dbc14)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:08.516980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:08.557681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:08.617129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80020e3533c24478891af4bb2c4dbc14'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.628055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80020e3533c24478891af4bb2c4dbc14'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.645931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80020e3533c24478891af4bb2c4dbc14'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.646778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80020e3533c24478891af4bb2c4dbc14'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.654266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80020e3533c24478891af4bb2c4dbc14'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:08.660574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1124/2269] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:08 2022] DELETE /identity/v3/users/80020e3533c24478891af4bb2c4dbc14 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:09.324731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.326767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.344014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f88c3b-78b3-460e-8758-082fe3b8f920 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.379868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.383667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.422192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.423014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.423320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.423560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] PATH_INFO: `/v3/projects/9ae97530f84542f49cb677cbfd47bf03` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.424923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae97530f84542f49cb677cbfd47bf03)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:09.426946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:09.473349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e3757-fd32-40d9-8243-8e5148891b3b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.484457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:09.498704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae97530f84542f49cb677cbfd47bf03', 'request_id': 'req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.508930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae97530f84542f49cb677cbfd47bf03', 'request_id': 'req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.516587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae97530f84542f49cb677cbfd47bf03', 'request_id': 'req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.530802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] The token cache is being invalidate because project 9ae97530f84542f49cb677cbfd47bf03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:09.531097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.531994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5aa1ce09-cf46-4655-9a20-1befcb2c0fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.533999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1146/2270] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:09 2022] DELETE /identity/v3/projects/9ae97530f84542f49cb677cbfd47bf03 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:09.540384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c71a0-6488-435d-9027-f85bd8795c64 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.540624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c71a0-6488-435d-9027-f85bd8795c64 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.540897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c71a0-6488-435d-9027-f85bd8795c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.550273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.551029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.551316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.551643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] PATH_INFO: `/v3/users/052daf02c895403780682b87894dc723` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.552855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052daf02c895403780682b87894dc723)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:09.554874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:09.564967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f06c71a0-6488-435d-9027-f85bd8795c64 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:09.569367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f06c71a0-6488-435d-9027-f85bd8795c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.581294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1147/2271] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:09.592663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.592942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.593220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.614608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052daf02c895403780682b87894dc723'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.616944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:09.617812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.619294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:09.626469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052daf02c895403780682b87894dc723'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.627681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1148/2272] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:09.636159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.646557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849714fb-21ac-45ae-9ab0-ba017e126a2c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.647986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052daf02c895403780682b87894dc723'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.648986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052daf02c895403780682b87894dc723'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.652965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.653692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.654040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.654402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.655604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:09.656104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:09.659015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5c1afc41-6551-4bb0-86b8-d3581bff5544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052daf02c895403780682b87894dc723'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:09.659787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:09.666937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1125/2273] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:09 2022] DELETE /identity/v3/users/052daf02c895403780682b87894dc723 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:09.671679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.672290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.672794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.676743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1149/2274] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:00:09.681826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.683815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.703266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bfb5d-1e79-4e54-9995-eaa78b1872ae admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.706875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:09.708658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.786811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.787981 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.788709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.789202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.790109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:09.794055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:09.801996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1126/2275] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:09.815962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.825620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:09.837904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1150/2276] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:09 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:09.843242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:09.912100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362b2f-221b-49ab-b367-b13567cf52b7 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.958310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c5a50-9fb4-40df-bd47-7ce1ce3540df admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:09.965886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.966584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.966871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.967137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:09.967928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:09.970343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:09.994912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:09.995686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:09.995966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:09.996235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.013156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.016040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1151/2277] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:09 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:10.019771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.023731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.033081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1127/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:10.038907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935015c-71bc-4dbb-869c-aebb804b2987 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.044862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.045578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.045916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.046249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.047007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.048880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.061786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1152/2279] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:10.066074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.076656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3ac05-4dcd-4da8-8121-084763ce3e24 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.083159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.084150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.084542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.084850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.085504 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.087199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52ab5f-7487-4db9-8f60-5443305d2081 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.099807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1128/2280] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:00:10.105454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.116775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a46a1-d20f-444e-975a-7588b70c3170 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.121874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.122667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.122952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.123230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] PATH_INFO: `/v3/projects/927d0b6338fd43348384d14fdbf7c561/users/595ed83fafd94297a20d6dd491f61474/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.126240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:10.145666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] RBAC: Authorizing `identity:create_grant(project_id=927d0b6338fd43348384d14fdbf7c561, user_id=595ed83fafd94297a20d6dd491f61474, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.148394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.166273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1153/2281] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:10 2022] PUT /identity/v3/projects/927d0b6338fd43348384d14fdbf7c561/users/595ed83fafd94297a20d6dd491f61474/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:10.177298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.178110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.178413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.225728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] MFA Rules not processed for user `595ed83fafd94297a20d6dd491f61474`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:10.226973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:10.231902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:10.311926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1129/2282] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:10.325369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.327306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:10.363540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.364988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:10.405073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1721b9-08c2-4581-bff0-c5403c6164db admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.411155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.412051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.412224 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.412503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.423253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.424691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.434546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1154/2283] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:10.461430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec0909-d719-42d6-b53b-6d5bef2b5afa None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.468781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.469507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.469788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.470062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] PATH_INFO: `/v3/projects/828be2d6209d4d7d9d2b5b5822483bed` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.471262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828be2d6209d4d7d9d2b5b5822483bed)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.473205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.533499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828be2d6209d4d7d9d2b5b5822483bed', 'request_id': 'req-86815fcf-4894-4f3a-a77f-e3fe29e51d19'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.548758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828be2d6209d4d7d9d2b5b5822483bed', 'request_id': 'req-86815fcf-4894-4f3a-a77f-e3fe29e51d19'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.555810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828be2d6209d4d7d9d2b5b5822483bed', 'request_id': 'req-86815fcf-4894-4f3a-a77f-e3fe29e51d19'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.584307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] The token cache is being invalidate because project 828be2d6209d4d7d9d2b5b5822483bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:10.584911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.585756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.591209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1130/2284] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:10 2022] DELETE /identity/v3/projects/828be2d6209d4d7d9d2b5b5822483bed => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:10.595454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:10.611185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc457a-a92e-4c15-99fa-70e1fe536b66 service glance] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:10.676096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:10.677015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:10.677503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:10.677769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] PATH_INFO: `/v3/users/64ccda3e2ce94ae5b8c904bb6dbd29a2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:10.679008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ccda3e2ce94ae5b8c904bb6dbd29a2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:10.681070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:10.746578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ccda3e2ce94ae5b8c904bb6dbd29a2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.755716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ccda3e2ce94ae5b8c904bb6dbd29a2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.774304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ccda3e2ce94ae5b8c904bb6dbd29a2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.775137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ccda3e2ce94ae5b8c904bb6dbd29a2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.792661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0f46bfe9-d7b4-4b48-98e2-71c1d8971f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ccda3e2ce94ae5b8c904bb6dbd29a2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:10.801674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1155/2285] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:10 2022] DELETE /identity/v3/users/64ccda3e2ce94ae5b8c904bb6dbd29a2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:11.214594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.216342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:11.232765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86815fcf-4894-4f3a-a77f-e3fe29e51d19 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.275072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:11.318881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.319548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.319830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.320101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] PATH_INFO: `/v3/projects/7a4d4ba81e704583967ee816641c8b07` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.321373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4d4ba81e704583967ee816641c8b07)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.323510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.379604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4d4ba81e704583967ee816641c8b07', 'request_id': 'req-0aed9862-2ddf-446f-82ce-77b04b5443fb'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:11.385260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4d4ba81e704583967ee816641c8b07', 'request_id': 'req-0aed9862-2ddf-446f-82ce-77b04b5443fb'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:11.392025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4d4ba81e704583967ee816641c8b07', 'request_id': 'req-0aed9862-2ddf-446f-82ce-77b04b5443fb'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:11.408181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] The token cache is being invalidate because project 7a4d4ba81e704583967ee816641c8b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:11.408447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:11.409198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0aed9862-2ddf-446f-82ce-77b04b5443fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:11.410771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1131/2286] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:11 2022] DELETE /identity/v3/projects/7a4d4ba81e704583967ee816641c8b07 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:11.416266 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.416872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.416872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.428689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:11.448954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:11.450780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:11.522281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1156/2287] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:11.528239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.528525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.528815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.552407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:11.554084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:11.563583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1132/2288] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:11.568606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.581454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7078a-58f5-4c42-ad25-f937870e3ac8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.587618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.588756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.589081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.589332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.590058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:11.590374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.592331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.611219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1157/2289] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:00:11.612463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.640288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a5d5e-006c-496e-9a75-5b27195ccb5e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.646238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.647000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.647346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.647625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.648280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.650211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.686143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1133/2290] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:11.688332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.702290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a027c1-8f86-4ec9-b333-e9d79ad218a2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.783861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.784496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.784782 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.785055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.785702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.787766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.818404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1158/2291] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:11.821589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.833367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdf6d-9c9f-4ae0-a0c0-3bbb16915d90 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.839960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.840935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.841190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.841381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.842044 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.843817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.855364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1134/2292] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:00:11.859693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.871257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a084c9-a7dd-4e66-a1a4-b99a532374d5 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.877717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.878569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.878939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.879321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.880074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.881801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933427a-aaf0-43a5-83cf-7d923d49662c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.894590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1159/2293] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:00:11.899036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:11.909260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782a2cf-7ac0-4bb4-9f41-f082bf1eff4c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:11.914131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:11.914868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.915126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.915394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] PATH_INFO: `/v3/projects/2b7e85d04148461586344eaee4908045/users/e70c6401ac7a4b29b5b77935bc2d0391/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.937236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7e85d04148461586344eaee4908045, user_id=e70c6401ac7a4b29b5b77935bc2d0391, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:11.940785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:11.969321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1135/2294] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:11 2022] PUT /identity/v3/projects/2b7e85d04148461586344eaee4908045/users/e70c6401ac7a4b29b5b77935bc2d0391/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:11.981897 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.982611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.983438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.057718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] MFA Rules not processed for user `e70c6401ac7a4b29b5b77935bc2d0391`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:12.059309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:12.148431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:12.159711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1160/2295] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:12.160723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.322093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de95dc7-6b59-4d76-8413-daa8d4d4dd00 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.366554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.366554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.366554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.366554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.367106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.367106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.384275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:12.421491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.427942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1136/2296] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:12 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:12.452015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd1359-7b8f-4a83-9712-763b94dd005d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.529350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.530305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.530811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.531514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.533114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.535436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.577438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1161/2297] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:12 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:12.581162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.605361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42b26-695f-446f-ab03-846ba0c0a678 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.611988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.612719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.613289 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.613399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.614131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.615893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.661569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1137/2298] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:12.665736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.676643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab043cc-8213-4292-ad08-642f69d8d70d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.682743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.683706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.684164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.684629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] PATH_INFO: `/v3/projects/30546cd5cd6d4848892d58b1061be368/users/89381350389942eb98532e05d840230f/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.705340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=30546cd5cd6d4848892d58b1061be368, user_id=89381350389942eb98532e05d840230f, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.709055 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.737639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1162/2299] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:12 2022] PUT /identity/v3/projects/30546cd5cd6d4848892d58b1061be368/users/89381350389942eb98532e05d840230f/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:12.741933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.756075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938502f1-694a-4246-89be-ba966513bddd admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.836324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.837062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.837546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.839152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.839275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.842242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a286bbc-d4d3-4f03-bd8b-f0377ad323c5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.859157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1138/2300] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:12.859714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.872729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c66f-ea8e-40b8-abdf-f8e2aa89f8cb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.877956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.878786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.879163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.879540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] PATH_INFO: `/v3/projects/30546cd5cd6d4848892d58b1061be368/users/89381350389942eb98532e05d840230f/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.881939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30546cd5cd6d4848892d58b1061be368, user_id=89381350389942eb98532e05d840230f, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.884778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.906450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1163/2301] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:12 2022] PUT /identity/v3/projects/30546cd5cd6d4848892d58b1061be368/users/89381350389942eb98532e05d840230f/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:12.911102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.911550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.911985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.949085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] MFA Rules not processed for user `89381350389942eb98532e05d840230f`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:12.950278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:13.047488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1139/2302] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:13.058256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:13.060095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:13.135333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cae55-854b-4ce3-a796-1c59561ea4c3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:13.140479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:13.148873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:13.149369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:13.150712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:13.170263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:13.171567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:13.189164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1164/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:13.314094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:13.326682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a640b0a-3075-4a87-8481-707ba5ae0ea3 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:13.334360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:13.341686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:13.342478 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:13.342753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:13.436506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:13.451089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:13.453887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bc635-1382-4e3b-8694-97ec5af8107c service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:13.502862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1140/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:00:14.069232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:14.071581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:14.149345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:14.155314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0c708-fae8-4cfc-9f00-5fa7f08f18c8 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:14.161282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:14.162035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:14.162316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:14.162621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] PATH_INFO: `/v3/users/014ea7a1f8124a6b83083e367c0a1db2` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:14.163941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014ea7a1f8124a6b83083e367c0a1db2)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:14.167352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:14.243296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014ea7a1f8124a6b83083e367c0a1db2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:14.253679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014ea7a1f8124a6b83083e367c0a1db2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:14.292252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014ea7a1f8124a6b83083e367c0a1db2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:14.296424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014ea7a1f8124a6b83083e367c0a1db2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:14.316693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014ea7a1f8124a6b83083e367c0a1db2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:14.326046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1165/2305] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:14 2022] DELETE /identity/v3/users/014ea7a1f8124a6b83083e367c0a1db2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:14.335450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b24aa4-440f-4d28-a0ac-5b1d9b1221eb None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:14.336541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b24aa4-440f-4d28-a0ac-5b1d9b1221eb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:14.336790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b24aa4-440f-4d28-a0ac-5b1d9b1221eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:14.366666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-43b24aa4-440f-4d28-a0ac-5b1d9b1221eb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:14.368061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-43b24aa4-440f-4d28-a0ac-5b1d9b1221eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:14.453608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1141/2306] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:14.468893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:14.471106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:14.580131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0eae6-92cb-4764-9b4a-d929a739b1d3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:14.593285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:14.594205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:14.594753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:14.595154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:14.620712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:14.622889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:14.639587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1166/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:14.990870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b87f-9a08-4a14-9e20-3b77dc1d50ae None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:14.991609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b87f-9a08-4a14-9e20-3b77dc1d50ae None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:14.992038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5b87f-9a08-4a14-9e20-3b77dc1d50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.027529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e5e5b87f-9a08-4a14-9e20-3b77dc1d50ae None None] MFA Rules not processed for user `b5b22eed68ce4d879de69b94084f1fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:15.029063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e5e5b87f-9a08-4a14-9e20-3b77dc1d50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:15.065477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.067791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:15.097046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776de8fe-b0dd-4136-9508-48815bd7c4c4 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.109893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.110894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.112270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.112818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] PATH_INFO: `/v3/projects/3f5cee44d2f240c6bd8b5c538fc13956` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.115665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5cee44d2f240c6bd8b5c538fc13956)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.120826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.130548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1142/2308] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:15.233968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5cee44d2f240c6bd8b5c538fc13956', 'request_id': 'req-93d04b72-e62d-4d9f-8f6f-9a559859695d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:15.244916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5cee44d2f240c6bd8b5c538fc13956', 'request_id': 'req-93d04b72-e62d-4d9f-8f6f-9a559859695d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:15.256887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5cee44d2f240c6bd8b5c538fc13956', 'request_id': 'req-93d04b72-e62d-4d9f-8f6f-9a559859695d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:15.271649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:15.281740 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] The token cache is being invalidate because project 3f5cee44d2f240c6bd8b5c538fc13956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:15.282199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:15.283532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-93d04b72-e62d-4d9f-8f6f-9a559859695d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:15.289269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1167/2309] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:15 2022] DELETE /identity/v3/projects/3f5cee44d2f240c6bd8b5c538fc13956 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:15.291098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.291405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.291631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.319446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:15.320581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:15.453243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1143/2310] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:15.469373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.469887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.470950 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.493271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:15.494662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:15.503491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1168/2311] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:15.507779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.518998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede290a-db46-4026-9296-7e28226ec339 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.524665 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.525308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.525594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.525872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.526578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:15.526891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.528551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.549629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1144/2312] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:00:15.554553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.565228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70416d2-6f50-459e-ab3e-9fddaa2b8e18 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.571207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.571856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.574072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.574072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.574072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.575966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.618567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1169/2313] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:15 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:15.624249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.637357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36d76e-cf57-4420-b89a-9f025daf61c9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.669726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:15.707861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.708491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.708765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.709205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.709880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.712624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.739863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1145/2314] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:15 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:15.744610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.761644 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40be9c-462a-44b3-a0ba-130f6cd4364c admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.767461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.768235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.768487 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.768755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.769417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.771274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.785501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1170/2315] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:15.787379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.799451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bdf2-e49d-4ae7-b897-900eb68d35ad admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.806933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.807606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.807937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.808246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.808990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.811008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56846f6e-bee9-41d7-9231-8e50ee3f215b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.827680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1146/2316] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:15.829126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:15.840534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3689-3fe7-4393-9e6a-3db386445bab admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:15.845834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:15.846943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.847407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.847894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] PATH_INFO: `/v3/projects/f715534d8f0543d08a5ee41fe2cb9e80/users/83e4c3e9f5114abaac61246b37961924/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.864986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f715534d8f0543d08a5ee41fe2cb9e80, user_id=83e4c3e9f5114abaac61246b37961924, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:15.868317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:15.887311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1171/2317] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:15 2022] PUT /identity/v3/projects/f715534d8f0543d08a5ee41fe2cb9e80/users/83e4c3e9f5114abaac61246b37961924/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:15.910492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:15.910870 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:15.911569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:15.952259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] MFA Rules not processed for user `83e4c3e9f5114abaac61246b37961924`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:15.953647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.028640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1147/2318] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:16.040533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:16.042356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.104849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028307c-8a9c-4d41-9388-fb2565e0b9ec admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:16.109443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:16.110095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:16.110390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:16.110676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:16.120250 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:16.121689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41fc6f1-ab6f-4628-9f58-9c8bafdf7765 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:16.131692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1172/2319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:16.508910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:16.510580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.601118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6b3c4-025c-4cef-a0dd-feb93fd55ce9 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:16.605772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:16.606483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:16.606762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:16.607034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] PATH_INFO: `/v3/users/e70c6401ac7a4b29b5b77935bc2d0391` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:16.608200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70c6401ac7a4b29b5b77935bc2d0391)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:16.610328 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:16.668681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:16.676728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70c6401ac7a4b29b5b77935bc2d0391'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:16.687129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70c6401ac7a4b29b5b77935bc2d0391'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:16.700427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70c6401ac7a4b29b5b77935bc2d0391'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:16.701237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70c6401ac7a4b29b5b77935bc2d0391'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:16.711788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70c6401ac7a4b29b5b77935bc2d0391'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:16.719351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1148/2320] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:16 2022] DELETE /identity/v3/users/e70c6401ac7a4b29b5b77935bc2d0391 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:16.724697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:16.725054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:16.725404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:16.747282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:16.748534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.822335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1173/2321] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:16.832945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:16.834484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.911949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c9b99-82d5-4d86-bf87-390f728523b5 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:16.916687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:16.917256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:16.917482 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:16.918114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:16.927965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:16.929423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:16.940318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1149/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:17.488942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:17.490813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:17.513438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00a8fd-760a-4ad8-9703-b7c5fa088775 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:17.525935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:17.526961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:17.527317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:17.527730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] PATH_INFO: `/v3/projects/2b7e85d04148461586344eaee4908045` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:17.530575 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7e85d04148461586344eaee4908045)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:17.535441 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:17.623526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:17.639751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7e85d04148461586344eaee4908045', 'request_id': 'req-85502b6b-6707-469f-98e8-e5ad98205624'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.647245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7e85d04148461586344eaee4908045', 'request_id': 'req-85502b6b-6707-469f-98e8-e5ad98205624'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.656621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7e85d04148461586344eaee4908045', 'request_id': 'req-85502b6b-6707-469f-98e8-e5ad98205624'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.675571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] The token cache is being invalidate because project 2b7e85d04148461586344eaee4908045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:17.676109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.676886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.678891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1174/2323] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:17 2022] DELETE /identity/v3/projects/2b7e85d04148461586344eaee4908045 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:17.682857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:17.694047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e321-9266-400a-811d-fb31de84a08b service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:17.799892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:17.800809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:17.801401 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:17.801763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] PATH_INFO: `/v3/users/89381350389942eb98532e05d840230f` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:17.803337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89381350389942eb98532e05d840230f)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:17.805548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:17.873983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89381350389942eb98532e05d840230f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.883402 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89381350389942eb98532e05d840230f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.889559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:17.899508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89381350389942eb98532e05d840230f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.900407 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89381350389942eb98532e05d840230f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.911159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-1aca104d-0e6e-412e-891e-5a7b389a0c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89381350389942eb98532e05d840230f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:17.918817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1150/2324] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:17 2022] DELETE /identity/v3/users/89381350389942eb98532e05d840230f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:18.229554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.230879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:18.253104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85502b6b-6707-469f-98e8-e5ad98205624 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.320412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.321160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.321437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.321705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] PATH_INFO: `/v3/projects/30546cd5cd6d4848892d58b1061be368` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.322928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30546cd5cd6d4848892d58b1061be368)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.324907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.373165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30546cd5cd6d4848892d58b1061be368', 'request_id': 'req-ae3f48f0-1e21-41e6-8630-10667a027099'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:18.378702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30546cd5cd6d4848892d58b1061be368', 'request_id': 'req-ae3f48f0-1e21-41e6-8630-10667a027099'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:18.385101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30546cd5cd6d4848892d58b1061be368', 'request_id': 'req-ae3f48f0-1e21-41e6-8630-10667a027099'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:18.400868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] The token cache is being invalidate because project 30546cd5cd6d4848892d58b1061be368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:18.401160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:18.401773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ae3f48f0-1e21-41e6-8630-10667a027099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:18.403325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1175/2325] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:18 2022] DELETE /identity/v3/projects/30546cd5cd6d4848892d58b1061be368 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:18.408432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.408722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.408999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.430944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:18.432218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:18.499217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1151/2326] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:18.507511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.508020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.508020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.528481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:18.529773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:18.538044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1176/2327] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:18.542538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.552492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bd8e7-cee4-4915-b86f-dfefeb3eea9b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.557786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.558651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.558893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.559167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.559848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:18.560156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.561991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.576523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1152/2328] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:00:18.580976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.591858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f99bce-6203-4287-ab83-270fa2900860 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.597657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.598260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.598801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.599070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.599706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.601796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.646632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1177/2329] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:18 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:18.650954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.662217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734e21c-e123-4494-aaa2-77e021c318fb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.726869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.727563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.727742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.728006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.728653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.730610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.754966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1153/2330] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:18 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:18.761076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.771873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf3930-086b-43dc-9a95-720ad96efffa admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.777054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.777652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.777929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.778193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.778854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.780457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.790745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1178/2331] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:18.794973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.806148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a0c40-070f-4d50-8e2c-d1326dd02bf1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.814560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.815254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.815558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.816092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.816841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.819356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa443f-5835-4121-baf4-e17fbb486274 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.832193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1154/2332] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:18.837267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:18.849007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a86bad-00b5-47da-8d8a-c1a0899261a8 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:18.854686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:18.855658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.856002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.856337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] PATH_INFO: `/v3/projects/9b75d6c88900432cb2e42eb15d291f5d/users/d12ed326b7b745ddbcc0e7694d38f729/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.869518 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:18.873732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b75d6c88900432cb2e42eb15d291f5d, user_id=d12ed326b7b745ddbcc0e7694d38f729, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:18.876814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:18.895663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1179/2333] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:18 2022] PUT /identity/v3/projects/9b75d6c88900432cb2e42eb15d291f5d/users/d12ed326b7b745ddbcc0e7694d38f729/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:18.906553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:18.906889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:18.907128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:18.935117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] MFA Rules not processed for user `d12ed326b7b745ddbcc0e7694d38f729`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:18.936345 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:18.977926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:19.016655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1155/2334] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:19.021170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.091043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66fee-7758-4dad-bb88-393805c4b2da admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.096530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.097162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.097533 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.097811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.098517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.100488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.133708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1180/2335] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:19 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:19.138239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.150334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141cbd88-4dcd-4b82-a6b1-a3a59a8bbb32 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.212500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.213465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.213765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.214034 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.214712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.216706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.240602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1156/2336] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:19 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:19.244885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.261187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7d4b7-56fe-4d11-b1a9-eea95f7990c6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.266995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.267567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.267891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.268245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.269187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.271689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.286102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1181/2337] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:19.290553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.301653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5c0ad-af0d-43e8-b168-712b99552e24 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.306872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.307568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.307826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.308126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] PATH_INFO: `/v3/projects/7441a6a8ae4a47949f81c8941a18812b/users/152f97c099894538b22ff4547855a6f4/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.324634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7441a6a8ae4a47949f81c8941a18812b, user_id=152f97c099894538b22ff4547855a6f4, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.327388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.345491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1157/2338] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:19 2022] PUT /identity/v3/projects/7441a6a8ae4a47949f81c8941a18812b/users/152f97c099894538b22ff4547855a6f4/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:19.349739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.359963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8621e-834a-4ed4-8014-3d8323c737aa admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.418078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.418867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.419004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.419274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.419942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.421549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385968c4-43c0-4df0-a32e-c867cf468e9a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.432098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1182/2339] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:19.436314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.447287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a7b8-5823-43c3-b3c5-5bea9cd0eb12 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.452579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.453363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.453601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.453903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] PATH_INFO: `/v3/projects/7441a6a8ae4a47949f81c8941a18812b/users/152f97c099894538b22ff4547855a6f4/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.456068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7441a6a8ae4a47949f81c8941a18812b, user_id=152f97c099894538b22ff4547855a6f4, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.458898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.470703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1158/2340] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:19 2022] PUT /identity/v3/projects/7441a6a8ae4a47949f81c8941a18812b/users/152f97c099894538b22ff4547855a6f4/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:19.481933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.482301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.482666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.512849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] MFA Rules not processed for user `152f97c099894538b22ff4547855a6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:19.514064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:19.561695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:19.586690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1183/2341] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:19.598116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:19.600220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:19.664183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676242-6fec-4aca-a3f5-7201d6b9b0da admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:19.667893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:19.668506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:19.668779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:19.669045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:19.712939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:19.741113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:19.742892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a77a3-d7f6-4579-b46d-d30a14140d14 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:19.752941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1159/2342] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:27.730282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:27.805129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddd1dc-2792-432d-a930-7a09f3e2c506 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:27.809971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:27.810751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:27.810911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:27.811169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] PATH_INFO: `/v3/users/d12ed326b7b745ddbcc0e7694d38f729` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:27.813219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12ed326b7b745ddbcc0e7694d38f729)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:27.815268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:27.888701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12ed326b7b745ddbcc0e7694d38f729'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:27.899304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12ed326b7b745ddbcc0e7694d38f729'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:27.915695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12ed326b7b745ddbcc0e7694d38f729'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:27.916685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12ed326b7b745ddbcc0e7694d38f729'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:27.925130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12ed326b7b745ddbcc0e7694d38f729'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:27.932278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1184/2343] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:27 2022] DELETE /identity/v3/users/d12ed326b7b745ddbcc0e7694d38f729 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:27.938550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:27.939308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:27.939308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:27.962067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:27.963722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:28.032122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1160/2344] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:28.041789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:28.043389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:28.110744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c77fe-a008-48c1-b07e-acb99fbda7c6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:28.115154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:28.115855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:28.116637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:28.116637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:28.128984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:28.130868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:28.156512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1185/2345] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:28.892163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:28.894489 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:28.917817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403dd05-1371-4f5d-bfea-73ffc856ce1b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:28.925987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:28.927051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:28.927529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:28.928047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] PATH_INFO: `/v3/projects/9b75d6c88900432cb2e42eb15d291f5d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:28.929787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b75d6c88900432cb2e42eb15d291f5d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:28.932578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:29.014755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b75d6c88900432cb2e42eb15d291f5d', 'request_id': 'req-6f533578-7822-4dfc-94bb-c350618be89f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.026823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b75d6c88900432cb2e42eb15d291f5d', 'request_id': 'req-6f533578-7822-4dfc-94bb-c350618be89f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.036088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b75d6c88900432cb2e42eb15d291f5d', 'request_id': 'req-6f533578-7822-4dfc-94bb-c350618be89f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.061323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] The token cache is being invalidate because project 9b75d6c88900432cb2e42eb15d291f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:29.061913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.062886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.064994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1161/2346] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:28 2022] DELETE /identity/v3/projects/9b75d6c88900432cb2e42eb15d291f5d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:29.069203 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:29.080475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de023f-94f9-490b-93d7-ea4a70fefc60 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:29.086707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:29.175962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:29.177135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:29.177701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:29.178116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] PATH_INFO: `/v3/users/152f97c099894538b22ff4547855a6f4` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:29.180279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] RBAC: Authorizing `identity:delete_user(user_id=152f97c099894538b22ff4547855a6f4)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:29.182845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:29.276887 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152f97c099894538b22ff4547855a6f4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.291136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152f97c099894538b22ff4547855a6f4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.307192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152f97c099894538b22ff4547855a6f4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.308223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152f97c099894538b22ff4547855a6f4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.316476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ca47933d-5df7-447b-814b-b51ebd7c347e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152f97c099894538b22ff4547855a6f4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.327505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1186/2347] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:29 2022] DELETE /identity/v3/users/152f97c099894538b22ff4547855a6f4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:29.647778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:29.649382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:29.671395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f533578-7822-4dfc-94bb-c350618be89f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:29.767040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:29.767889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:29.768566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:29.768904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] PATH_INFO: `/v3/projects/7441a6a8ae4a47949f81c8941a18812b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:29.770330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7441a6a8ae4a47949f81c8941a18812b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:29.772811 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:29.781653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:29.862198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7441a6a8ae4a47949f81c8941a18812b', 'request_id': 'req-d345ba2b-9d70-4368-879c-5dc7308a3ed7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.869943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7441a6a8ae4a47949f81c8941a18812b', 'request_id': 'req-d345ba2b-9d70-4368-879c-5dc7308a3ed7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.895674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7441a6a8ae4a47949f81c8941a18812b', 'request_id': 'req-d345ba2b-9d70-4368-879c-5dc7308a3ed7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.934924 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] The token cache is being invalidate because project 7441a6a8ae4a47949f81c8941a18812b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:29.935340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.936136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-d345ba2b-9d70-4368-879c-5dc7308a3ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:29.938422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1162/2348] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:29 2022] DELETE /identity/v3/projects/7441a6a8ae4a47949f81c8941a18812b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:29.943479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:29.943976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:29.944437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:29.983853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:29.985826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:30.070507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1187/2349] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:30.079493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.080685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.081201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.168095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:30.169450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:30.200962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.207238 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1163/2350] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:30.216686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362a5be-3fe4-4477-8f28-807c1dbc8ba8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.228456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.229200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.229598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.229983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.234421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:30.234421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.234421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.273146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1188/2351] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:00:30.283106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.296722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560e76-a8ed-4c2d-8949-4e6d8efb2332 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.303746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.304493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.305084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.305368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.306020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.311088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.379995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1164/2352] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:30 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:30.384527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.402011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a425688-de43-46e3-8b5c-1923669e63fc admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.417529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:30.512140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.512807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.513091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.513370 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.514048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.516229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.544110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1189/2353] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:30 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:30.548640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.561205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3c4ce-29d6-40c2-8ccb-cc45cebd3ebf admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.566481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.567135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.567413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.567684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.568353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.570009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.584978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1165/2354] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:30.590173 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.605655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3ac82-f1b3-44cc-8af6-a61fa4794c2a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.617235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.617874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.618150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.618448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.619104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.620768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff53e7b-176f-4aa9-85d9-eba120597d1b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.632812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1190/2355] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:30.637285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.648296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458abe5-8d46-4d31-9b76-4d6d52715131 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.652678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.653362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.653809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.654084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] PATH_INFO: `/v3/projects/d91250ab7b514c50ae571bcf59cb25dc/users/f98dc60883e84651a6388d6def7c3032/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.672896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91250ab7b514c50ae571bcf59cb25dc, user_id=f98dc60883e84651a6388d6def7c3032, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.676046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.692334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1166/2356] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:30 2022] PUT /identity/v3/projects/d91250ab7b514c50ae571bcf59cb25dc/users/f98dc60883e84651a6388d6def7c3032/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:30.704303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.704748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.705064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.739547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] MFA Rules not processed for user `f98dc60883e84651a6388d6def7c3032`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:30.740777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:30.828704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1191/2357] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:30.837846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:30.840532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:30.916106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fefd0-d03d-4e58-87ea-fa98cefa3cde admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:30.921213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:30.922033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:30.922463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:30.923680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:30.933732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:30.935340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60976178-160a-48e3-934e-bd1ef07167c1 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:30.948096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1167/2358] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:36.308168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:36.382099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f2a0-16f7-4c05-9dd9-25ebb307f40b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:36.386934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:36.387568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:36.387842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:36.388112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] PATH_INFO: `/v3/users/f98dc60883e84651a6388d6def7c3032` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:36.389316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98dc60883e84651a6388d6def7c3032)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:36.391391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:36.414241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:36.447618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98dc60883e84651a6388d6def7c3032'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:36.456564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98dc60883e84651a6388d6def7c3032'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:36.472459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98dc60883e84651a6388d6def7c3032'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:36.473357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98dc60883e84651a6388d6def7c3032'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:36.482039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98dc60883e84651a6388d6def7c3032'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:36.489227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1192/2359] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:36 2022] DELETE /identity/v3/users/f98dc60883e84651a6388d6def7c3032 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:36.494053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:36.495719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:36.495719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:36.520098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:36.521754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:36.567560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:36.618965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1168/2360] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:36.626185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:36.627891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:36.702110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57194354-3109-4f1c-bfe0-558fc7f15fbd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:36.709576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:36.710336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:36.710795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:36.711005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:36.743372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:36.745090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d55ba-b233-498b-a2cc-5435e41f64f2 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:36.790328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1193/2361] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:00:37.301148 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.302876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:37.315743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640170a-1598-4071-9d01-1eef32476a93 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.327035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.327715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.327995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.328266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] PATH_INFO: `/v3/projects/d91250ab7b514c50ae571bcf59cb25dc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.330637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91250ab7b514c50ae571bcf59cb25dc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.332663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.386473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91250ab7b514c50ae571bcf59cb25dc', 'request_id': 'req-5df6db04-3054-4f25-9fb2-66c93f220d5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:37.392453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91250ab7b514c50ae571bcf59cb25dc', 'request_id': 'req-5df6db04-3054-4f25-9fb2-66c93f220d5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:37.399435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91250ab7b514c50ae571bcf59cb25dc', 'request_id': 'req-5df6db04-3054-4f25-9fb2-66c93f220d5f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:37.413855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] The token cache is being invalidate because project d91250ab7b514c50ae571bcf59cb25dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:37.414146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:37.414785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5df6db04-3054-4f25-9fb2-66c93f220d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:37.416318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1169/2362] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:37 2022] DELETE /identity/v3/projects/d91250ab7b514c50ae571bcf59cb25dc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:37.421323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.421783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.421890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.441249 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:37.442463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:37.502841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1194/2363] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:37.510798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.511320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.511320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.533464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:37.534741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:37.547308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1170/2364] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:37.548962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.558923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc30877-1e46-48c5-95ae-cb609bcc1dad None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.563959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.564522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.564800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.565068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.565747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:37.566978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.567715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.582407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1195/2365] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:00:37.586968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.598050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bde026-3621-4b8a-8572-e15775efc5bb None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.603441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.604068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.604342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.604611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.605316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.607414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.636590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1171/2366] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:37 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:37.640899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.652277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3adb1-dfc7-48fe-9d43-710085a1f7b6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.686566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:37.733678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.734323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.734645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.734907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.735593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.737676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.765852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1196/2367] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:37 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:00:37.770886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.787240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406e1af-237c-4cb2-b332-a83f6f7a1d04 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.792156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.792780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.793064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.793338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.793994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.795647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.809548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1172/2368] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:37.814443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.829747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b3d70-b31a-41ff-bd12-5324f4fb7d4a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.836082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.836784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.836996 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.837765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.838472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.840085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73ff34-67e4-41c5-9d8b-cb9d1739144c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.857386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1197/2369] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:37.861909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:37.875119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b173e5-fbc3-4c72-b90d-f710da2b630f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:37.880832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:37.882013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.882438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.882896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] PATH_INFO: `/v3/projects/39ea215c38b643b790997c69eabd501b/users/28b86d8f0b5540f3adf8f4d9e3055e32/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.903260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ea215c38b643b790997c69eabd501b, user_id=28b86d8f0b5540f3adf8f4d9e3055e32, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:37.906135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:37.922886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1173/2370] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:37 2022] PUT /identity/v3/projects/39ea215c38b643b790997c69eabd501b/users/28b86d8f0b5540f3adf8f4d9e3055e32/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:37.934642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:37.935003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:37.935343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:37.971176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] MFA Rules not processed for user `28b86d8f0b5540f3adf8f4d9e3055e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:37.972727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:38.056453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1198/2371] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:38.067880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:38.070652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:38.145468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eedf0a-1c06-4583-b883-e9c45b896230 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:38.152146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:38.153096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:38.153372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:38.154127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:38.167947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:38.170753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:38.172863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94920460-9bd5-4dff-b700-5d8b686ef663 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:38.186031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1174/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:39.786071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:39.787911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:39.868367 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fc079-938e-4d32-b1a5-2d48fe206174 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:39.873189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:39.873842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:39.874116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:39.874416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] PATH_INFO: `/v3/users/83e4c3e9f5114abaac61246b37961924` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:39.875610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e4c3e9f5114abaac61246b37961924)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:39.877542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:39.920723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:39.926433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e4c3e9f5114abaac61246b37961924'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:39.934492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e4c3e9f5114abaac61246b37961924'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:39.950552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e4c3e9f5114abaac61246b37961924'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:39.951336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e4c3e9f5114abaac61246b37961924'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:39.959053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e4c3e9f5114abaac61246b37961924'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:39.965929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1199/2373] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:00:39 2022] DELETE /identity/v3/users/83e4c3e9f5114abaac61246b37961924 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:39.970809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:39.971308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:39.971549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:39.992395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:39.993708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:40.061069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1175/2374] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:40.071859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:40.074311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:40.137630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f51a-c380-4dcf-b58b-80608c879f0a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:40.141931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:40.142655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.142828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.143095 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.153194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:40.154658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1339fa-3239-4d34-bcd8-a0c7da1ea841 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:40.164576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1200/2375] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:40.489074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:40.490575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:40.502604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4ddf-583c-4b20-86a7-cbffce57b1d6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:40.507647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:40.508334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.508629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.509019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] PATH_INFO: `/v3/projects/f715534d8f0543d08a5ee41fe2cb9e80` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.510194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f715534d8f0543d08a5ee41fe2cb9e80)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:40.512405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:40.586054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f715534d8f0543d08a5ee41fe2cb9e80', 'request_id': 'req-f7e70898-d35d-4a35-b893-67cbbb83bf27'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:40.596125 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f715534d8f0543d08a5ee41fe2cb9e80', 'request_id': 'req-f7e70898-d35d-4a35-b893-67cbbb83bf27'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:40.617941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f715534d8f0543d08a5ee41fe2cb9e80', 'request_id': 'req-f7e70898-d35d-4a35-b893-67cbbb83bf27'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:40.635142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] The token cache is being invalidate because project f715534d8f0543d08a5ee41fe2cb9e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:00:40.635730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:40.636648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7e70898-d35d-4a35-b893-67cbbb83bf27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:00:40.638629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1176/2376] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:00:40 2022] DELETE /identity/v3/projects/f715534d8f0543d08a5ee41fe2cb9e80 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:00:40.643800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.644114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.644385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.666752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:40.667679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:40.735280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1201/2377] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:40.743710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.744012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.744283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.765755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:40.767405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:40.776209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1177/2378] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:40.785750 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:40.802484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a090b4-8a68-4ed4-8a30-95d659140bcb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:40.811708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:40.812629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.812908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.813196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.814719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:00:40.817195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:40.818953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:40.833484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1202/2379] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:00:40.839252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:40.849995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28f76a-424e-4002-aed9-c9b9750dd33d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:40.858159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:40.858797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.859443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.859702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.860431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:40.865224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:40.893004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1178/2380] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:00:40 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:40.897314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:40.907559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39946aa4-1dc2-4762-992c-70e66ba576ae admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:40.958214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:40.977161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:40.978182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:40.978702 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:40.979156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:40.980301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:40.984142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:41.015136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1203/2381] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:00:40 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:00:41.019750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:41.030781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24663f3a-e802-4990-a166-614eea86bd34 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:41.036002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:41.036639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:41.036939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:41.037147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:41.037803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:41.039936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:41.053697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1179/2382] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:41.055519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:41.067793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030df51-37e1-46ad-8ebb-dbff42bedbd4 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:41.072952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:41.073577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:41.073858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:41.074141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:41.074820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:41.076565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787301ed-4a9b-450f-9c94-868c8dfbe45b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:41.094135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1204/2383] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:00:41.098862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:41.109380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5094a4f-73f0-45e6-b1da-55ae6a2b4704 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:41.114051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:41.114753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:41.115036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:41.115306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] PATH_INFO: `/v3/projects/f676b613127f422f956fa36c08fa1723/users/d56e5f2da7df486299204b292ffc46da/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:41.130412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f676b613127f422f956fa36c08fa1723, user_id=d56e5f2da7df486299204b292ffc46da, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:41.133094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:41.150748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1180/2384] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:00:41 2022] PUT /identity/v3/projects/f676b613127f422f956fa36c08fa1723/users/d56e5f2da7df486299204b292ffc46da/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:00:41.163212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:41.163495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:41.163744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:41.199980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] MFA Rules not processed for user `d56e5f2da7df486299204b292ffc46da`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:41.202079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:41.294720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1205/2385] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:00:41.306770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:41.308531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:41.328580 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:41.386411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7a70d-141f-4bc3-81d4-a3557a6113f2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:41.391287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:41.391938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:41.392272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:41.392599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:41.403191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:41.404695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e7a29-2834-4252-99bc-ba89440aa0e9 service glance] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:41.415357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1181/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:51.291075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:51.410249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=19 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:13.358522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:13.361257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.473953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0914882-2656-4be2-82c9-e052935f7bd7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:13.483480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:13.484215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:13.484780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:13.485153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] PATH_INFO: `/v3/users/28b86d8f0b5540f3adf8f4d9e3055e32` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:13.486913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b86d8f0b5540f3adf8f4d9e3055e32)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:13.489631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:13.643670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6a69-0acf-485f-9e15-f1d843a6debd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:13.644247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6a69-0acf-485f-9e15-f1d843a6debd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:13.644247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e6a69-0acf-485f-9e15-f1d843a6debd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:13.670324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-263e6a69-0acf-485f-9e15-f1d843a6debd None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:13.672625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-263e6a69-0acf-485f-9e15-f1d843a6debd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.719059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b86d8f0b5540f3adf8f4d9e3055e32'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:13.730832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b86d8f0b5540f3adf8f4d9e3055e32'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:13.743204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1182/2387] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:13.746819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b86d8f0b5540f3adf8f4d9e3055e32'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:13.747697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b86d8f0b5540f3adf8f4d9e3055e32'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:13.757325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b86d8f0b5540f3adf8f4d9e3055e32'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:13.768508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1206/2388] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:13 2022] DELETE /identity/v3/users/28b86d8f0b5540f3adf8f4d9e3055e32 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:13.776064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:13.776416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:13.776758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:13.795020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:13.796958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.832656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:13.837344 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.916944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f71a9-d59c-476b-8026-3422f1f0b9f7 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:13.922882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:13.923576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:13.923862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:13.924136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:13.929592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.948237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1183/2389] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:13.959259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:13.961891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:13.974433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:14.013429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:14.015085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:14.024577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1207/2390] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:01:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:01:14.039922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e894a7-53d0-45c0-838a-bba4798b369a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:14.043748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:14.044368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:14.044653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:14.044922 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:14.056508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:14.059145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548957e-535c-4a7c-a590-8d9602a72227 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:14.070609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1184/2391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:01:14.690706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:14.704047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:14.730355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840b1eb-22f7-448c-96e5-213f9ed935bc service swift] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:14.737941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:14.739048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:14.739447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:14.739803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] PATH_INFO: `/v3/projects/39ea215c38b643b790997c69eabd501b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:14.741481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ea215c38b643b790997c69eabd501b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:14.743909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:14.815021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ea215c38b643b790997c69eabd501b', 'request_id': 'req-36f4f302-e28c-47f9-80fd-f826206022c0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:14.830323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ea215c38b643b790997c69eabd501b', 'request_id': 'req-36f4f302-e28c-47f9-80fd-f826206022c0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:14.830323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ea215c38b643b790997c69eabd501b', 'request_id': 'req-36f4f302-e28c-47f9-80fd-f826206022c0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:14.849472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] The token cache is being invalidate because project 39ea215c38b643b790997c69eabd501b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:01:14.849806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:14.850639 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-36f4f302-e28c-47f9-80fd-f826206022c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:14.854028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1208/2392] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:01:14 2022] DELETE /identity/v3/projects/39ea215c38b643b790997c69eabd501b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:14.859171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:14.859716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:14.859716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:14.884544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:14.885770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:14.988186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:15.017519 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1185/2393] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:15.025431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.025849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.025963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.052497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:15.054894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:15.070629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1209/2394] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:15.071648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.093040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f964be-4b1a-442b-94f8-81783b364bc1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.112743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.113507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.116483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.116973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.120515 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:01:15.121918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.127753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.154839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1186/2395] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:01:15.158921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.177759 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b78ef-2531-4e6e-8160-ae1489cc2ac4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.194623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.194623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.194623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.194623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.194965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.194965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.227929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1210/2396] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:01:15 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:01:15.232306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.243746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b19cf-e70a-4831-9078-48520d2d2110 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.353494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.354166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.354429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.354707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.356336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.358982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.404541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1187/2397] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:01:15 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:01:15.408853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.419932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad960-73b0-478c-b55c-b1449b0ccfab admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.424905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.425654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.426054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.426469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.427220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.429402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.447720 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1211/2398] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:15.448483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.462190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f2d88-fe11-476a-b09d-aead27f8f03a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.474774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.475501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.475784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.476059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.476717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.478357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7456b4b-b666-4725-87d1-70c462883f09 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.506434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1188/2399] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:15.509444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.522699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59fbb-7330-4741-9ea9-056f34fa6c86 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.543617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.544316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.544590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.544854 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] PATH_INFO: `/v3/projects/c401b957669e43bdabf08fa232b461be/users/66b7e2cda35c473597343822e622a3bd/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.562809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c401b957669e43bdabf08fa232b461be, user_id=66b7e2cda35c473597343822e622a3bd, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.572696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.606536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.606911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.607185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.632178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1212/2400] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:01:15 2022] PUT /identity/v3/projects/c401b957669e43bdabf08fa232b461be/users/66b7e2cda35c473597343822e622a3bd/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:15.665650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:15.667001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:15.802439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1189/2401] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:15.809217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:15.812699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:15.904732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:15.930843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de081-dfb4-4755-a5b8-7e2891f40b3b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:15.935992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:15.937795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:15.938080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:15.938354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:15.967815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:15.970272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0448762-03b7-440f-894e-69c755e3c767 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:15.996265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1213/2402] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:16.232904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:16.234480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:16.248940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae494b-5be6-4ad3-8513-824813f3d209 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:16.254084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:16.254997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:16.255331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:16.255745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] PATH_INFO: `/v3/users/595ed83fafd94297a20d6dd491f61474` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:16.257221 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=595ed83fafd94297a20d6dd491f61474)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:16.259754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:16.345611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595ed83fafd94297a20d6dd491f61474'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:16.348932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:16.359657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595ed83fafd94297a20d6dd491f61474'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:16.377979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595ed83fafd94297a20d6dd491f61474'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:16.378815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595ed83fafd94297a20d6dd491f61474'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:16.388324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595ed83fafd94297a20d6dd491f61474'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:16.400930 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:16.401379 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:16.401872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:16.404989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1190/2403] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:16 2022] DELETE /identity/v3/users/595ed83fafd94297a20d6dd491f61474 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:16.427200 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:16.429063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:16.516414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1214/2404] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:16.529881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:16.532262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:16.658013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542a6c-83f9-430f-b9c5-455e14cb40ab admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:16.663046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:16.663700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:16.664009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:16.664282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:16.690072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:16.691619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1023f0-fcaa-4153-b554-bee06855f539 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:16.708711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1191/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:17.672547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:17.675807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:17.703547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65585a7-effa-4ad4-82c4-cc05b8d2a2c3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:17.721387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:17.722060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:17.722335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:17.722637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] PATH_INFO: `/v3/projects/927d0b6338fd43348384d14fdbf7c561` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:17.723822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] RBAC: Authorizing `identity:delete_project(project_id=927d0b6338fd43348384d14fdbf7c561)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:17.727592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:17.856327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927d0b6338fd43348384d14fdbf7c561', 'request_id': 'req-4bfce09d-fc09-44ea-8123-8468e8a4812d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:17.875799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927d0b6338fd43348384d14fdbf7c561', 'request_id': 'req-4bfce09d-fc09-44ea-8123-8468e8a4812d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:17.887383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927d0b6338fd43348384d14fdbf7c561', 'request_id': 'req-4bfce09d-fc09-44ea-8123-8468e8a4812d'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:17.920040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] The token cache is being invalidate because project 927d0b6338fd43348384d14fdbf7c561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:01:17.920491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:17.921356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:17.923328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1215/2406] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:01:17 2022] DELETE /identity/v3/projects/927d0b6338fd43348384d14fdbf7c561 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:20.670056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:20.670680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:20.670680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:20.729605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:20.743430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:20.942999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1192/2407] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:01:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:20.965091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:20.967618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:21.156743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfce09d-fc09-44ea-8123-8468e8a4812d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:21.170886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:21.172350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:21.172830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:21.173279 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:21.189142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:21.201302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:21.206201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef8049-c014-4b30-ac1d-9215eb3f3e70 service swift] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:21.236498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1216/2408] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:01:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:01:21.757991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:21.760636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:22.015442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4eee5-7846-4379-8742-a80767e72583 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:22.028725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:22.031795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:22.032490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:22.032935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:22.039261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:22.080083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:22.082591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:22.110725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1193/2409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:24.466176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:24.466834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:24.466937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:24.556230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] MFA Rules not processed for user `66b7e2cda35c473597343822e622a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:24.557673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:24.724665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1217/2410] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:24.726588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:24.728208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:24.753446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:24.919733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963529f-a13c-4127-8edd-1c55568cee26 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:24.938340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:24.938936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:24.939373 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:24.940099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:24.940949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:24.945711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.097600 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1194/2411] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:01:24 2022] POST /identity/v3/projects => generated 385 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:01:25.106014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:25.126106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4de6b8-4d1e-43d3-855f-c3477d1459ea None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:25.312400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:25.313400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.313716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.315291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.316845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:25.326560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538635-1b0d-4fd8-a77f-221ffe6cdcfe admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.391500 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1218/2412] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:01:25 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:01:25.393551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:25.417309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.417544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18e8e-f9b0-479c-8abe-482e849d7c81 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:25.417640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.417876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.448016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:25.448966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.449474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.449901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.450730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:25.452386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.474346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] MFA Rules not processed for user `2577fc11946c4597857187ae7b464fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:25.477427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:25.486958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1195/2413] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:25.491395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:25.527852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036eb80-c7a7-4f30-b782-75b38537b3be admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:25.533069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:25.533879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.534229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.534630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] PATH_INFO: `/v3/projects/2d0806d18e154bf99b8672d168247c96/users/a7de0bda37784bb9a6717ff648317f5d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.587269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0806d18e154bf99b8672d168247c96, user_id=a7de0bda37784bb9a6717ff648317f5d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:25.590348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.636695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1219/2414] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:25.642761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1196/2415] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:01:25 2022] PUT /identity/v3/projects/2d0806d18e154bf99b8672d168247c96/users/a7de0bda37784bb9a6717ff648317f5d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:25.650114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:25.677297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132c66a-baa0-4001-a944-a6b7d50fad39 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:25.713732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:25.816866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:25.820179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.820814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.821155 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.822403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:25.825319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1c7cd-7468-4eb5-9053-e45c2b9e2eaf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.846815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1220/2416] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:25.851274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:25.867762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ea329-313d-4213-af72-af9d59d9c2f7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:25.891835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:25.892537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:25.892899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:25.893082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] PATH_INFO: `/v3/projects/2d0806d18e154bf99b8672d168247c96/users/a7de0bda37784bb9a6717ff648317f5d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:25.900058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0806d18e154bf99b8672d168247c96, user_id=a7de0bda37784bb9a6717ff648317f5d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:25.915586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:25.983270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1197/2417] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:01:25 2022] PUT /identity/v3/projects/2d0806d18e154bf99b8672d168247c96/users/a7de0bda37784bb9a6717ff648317f5d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:35.836678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=20 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:35.943881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:37.339843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5d96a-cdf7-41fb-8406-2acff5b68bb6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:37.341293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5d96a-cdf7-41fb-8406-2acff5b68bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:37.341798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5d96a-cdf7-41fb-8406-2acff5b68bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:37.406254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-45b5d96a-cdf7-41fb-8406-2acff5b68bb6 None None] MFA Rules not processed for user `a7de0bda37784bb9a6717ff648317f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:37.407566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-45b5d96a-cdf7-41fb-8406-2acff5b68bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:37.513534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1221/2418] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:37.527775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:37.530266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:37.620232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0f77-322b-4d70-94db-d4fcf66e3e61 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:37.624120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:37.624778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:37.625066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:37.625367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:37.707899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:37.710133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:37.721628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1198/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:37.925559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc063b-dcb3-49ef-85f2-490c21dded05 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:37.926202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc063b-dcb3-49ef-85f2-490c21dded05 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:37.926799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc063b-dcb3-49ef-85f2-490c21dded05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:37.957833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-debc063b-dcb3-49ef-85f2-490c21dded05 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:37.959906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-debc063b-dcb3-49ef-85f2-490c21dded05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:38.021739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1222/2420] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:01:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:38.043332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:38.192435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa317d-72e2-4c14-b3c2-be06ab97ffd3 service glance] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:38.207178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:38.208293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:38.208739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:38.209529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:38.246195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:38.247847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed067ff6-a3fd-48ed-8913-ad2b9f8e455b service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:38.269398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1199/2421] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:01:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:01:44.475695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3b00b-56c1-4901-8dde-3a2a6e553c06 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:44.476305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3b00b-56c1-4901-8dde-3a2a6e553c06 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:44.476802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3b00b-56c1-4901-8dde-3a2a6e553c06 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:44.481271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1223/2422] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 20:01:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:01:44.496259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:44.496869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:44.496869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:44.530026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:44.531629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:44.628807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:44.705101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1200/2423] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 20:01:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:45.439112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47c18c-6fae-4781-b34f-25975bcf645c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:45.440002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47c18c-6fae-4781-b34f-25975bcf645c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:45.440504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47c18c-6fae-4781-b34f-25975bcf645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:45.516931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6d47c18c-6fae-4781-b34f-25975bcf645c None None] MFA Rules not processed for user `7c17f29c73f643ca83a08f4e44e6279b`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:45.519282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6d47c18c-6fae-4781-b34f-25975bcf645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:45.750529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1224/2424] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:46.769603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:46.772005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:46.965237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df11a-7ed1-4ef7-b314-d7fee81fc74a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:46.970935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:46.970935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:46.971424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:46.971424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] PATH_INFO: `/v3/users/d56e5f2da7df486299204b292ffc46da` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:46.972528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56e5f2da7df486299204b292ffc46da)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:46.974678 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:47.068595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56e5f2da7df486299204b292ffc46da'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:47.085960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56e5f2da7df486299204b292ffc46da'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:47.121865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56e5f2da7df486299204b292ffc46da'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:47.123279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56e5f2da7df486299204b292ffc46da'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:47.142498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56e5f2da7df486299204b292ffc46da'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:47.164844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1201/2425] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:46 2022] DELETE /identity/v3/users/d56e5f2da7df486299204b292ffc46da => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:47.174973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611275-c699-43be-854b-246ee71ab959 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:47.176540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611275-c699-43be-854b-246ee71ab959 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:47.177158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611275-c699-43be-854b-246ee71ab959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:47.206514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-60611275-c699-43be-854b-246ee71ab959 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:47.208170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-60611275-c699-43be-854b-246ee71ab959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:47.253001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:47.315352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1225/2426] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:47.331481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:47.333782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:47.497622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5840e-df0b-4200-b31d-1532678ae3da admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:47.513062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:47.523053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:47.527131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:47.527427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:47.557267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:47.564249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:47.575443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:47.598999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1202/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:48.734818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611275-c699-43be-854b-246ee71ab959 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:48.734818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-60611275-c699-43be-854b-246ee71ab959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:48.748909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611275-c699-43be-854b-246ee71ab959 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:48.755955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:48.756730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:48.757274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:48.757745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] PATH_INFO: `/v3/projects/f676b613127f422f956fa36c08fa1723` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:48.759872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f676b613127f422f956fa36c08fa1723)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:48.762921 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:48.895000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f676b613127f422f956fa36c08fa1723', 'request_id': 'req-808af58c-603c-4886-8c88-300ac5061af9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:48.903554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f676b613127f422f956fa36c08fa1723', 'request_id': 'req-808af58c-603c-4886-8c88-300ac5061af9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:48.912582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f676b613127f422f956fa36c08fa1723', 'request_id': 'req-808af58c-603c-4886-8c88-300ac5061af9'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:48.949009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] The token cache is being invalidate because project f676b613127f422f956fa36c08fa1723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:01:48.949501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:48.950518 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-808af58c-603c-4886-8c88-300ac5061af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:48.953431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1226/2428] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:01:48 2022] DELETE /identity/v3/projects/f676b613127f422f956fa36c08fa1723 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:52.544468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:52.545934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:52.630950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409033a8-eeef-4569-b2d3-75b2313dc73f service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:52.707957 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:52.709347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:52.710258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:52.710258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:52.712507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:52.836544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:52.839862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0c45b-d9c7-44fb-8322-0cca75b9733f service glance] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:52.863460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1203/2429] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:53.090709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.091365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.091725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] PATH_INFO: `/` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.094107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1227/2430] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 20:01:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:01:53.103792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ab5c-8377-4a3e-bf1b-98a6d9493180 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.104438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ab5c-8377-4a3e-bf1b-98a6d9493180 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.105051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ab5c-8377-4a3e-bf1b-98a6d9493180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.148129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a676ab5c-8377-4a3e-bf1b-98a6d9493180 None None] MFA Rules not processed for user `1a83958d50934b24ae40da8760b2fb85`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:53.151769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a676ab5c-8377-4a3e-bf1b-98a6d9493180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:53.346733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1204/2431] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:53.350002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:53.351957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:53.366801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331c46b-2e15-4c96-b98a-066f0c09cfd4 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:53.374829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:53.375354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.375635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.375905 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.386904 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:53.388989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784cb04-cd91-4861-8ce3-9499444d4df3 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:53.406868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1228/2432] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:53.571256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.572101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.572101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] PATH_INFO: `/` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.573470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1205/2433] 10.222.0.52 () {54 vars in 921 bytes} [Tue Sep 13 20:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:01:53.589276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13963c-d151-44dd-b93b-ee5b53933867 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.589529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13963c-d151-44dd-b93b-ee5b53933867 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.589806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13963c-d151-44dd-b93b-ee5b53933867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.615967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6c13963c-d151-44dd-b93b-ee5b53933867 None None] MFA Rules not processed for user `463fcb701698473bad12e8101b9018ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:53.617302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6c13963c-d151-44dd-b93b-ee5b53933867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:53.642328 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1229/2434] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Sep 13 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:53.651812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:53.654860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:53.670839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:53.681132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c19c5c-ae1a-47a3-b5f7-178a4d5d8c12 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:53.688156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:53.688911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.689279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.689639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.715224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:53.716895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:53.745808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1206/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:01:53.802352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0cb2-9fcc-4d16-a8d9-caaeefabfb64 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:53.802843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0cb2-9fcc-4d16-a8d9-caaeefabfb64 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:53.803437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0cb2-9fcc-4d16-a8d9-caaeefabfb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:53.826048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-939d0cb2-9fcc-4d16-a8d9-caaeefabfb64 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:53.827622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-939d0cb2-9fcc-4d16-a8d9-caaeefabfb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:53.936972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1230/2436] 10.222.0.52 () {58 vars in 967 bytes} [Tue Sep 13 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:53.947154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:53.950035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:54.096093 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50550005-533c-4021-8123-07dc79e96d85 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:54.102190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:54.102885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:54.103163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:54.103431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:54.117685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:54.121116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:54.148005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1207/2437] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:01:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:01:54.374278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:54.375202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:54.375630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:54.402692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] MFA Rules not processed for user `66b7e2cda35c473597343822e622a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:54.403784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:54.501808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:54.551453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1231/2438] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:57.130095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:57.133024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:57.287541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0286307-86ac-4f16-9bdc-b236631e490d service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:57.294725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:57.303432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:57.303890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:57.304323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] PATH_INFO: `/v3/users/b5b22eed68ce4d879de69b94084f1fb8` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:57.306477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b22eed68ce4d879de69b94084f1fb8)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:57.309387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:57.451603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b22eed68ce4d879de69b94084f1fb8'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.468161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b22eed68ce4d879de69b94084f1fb8'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.485844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b22eed68ce4d879de69b94084f1fb8'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.487617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b22eed68ce4d879de69b94084f1fb8'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.499523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-19135f40-0aa3-4c5f-b174-e0079b97c6c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b22eed68ce4d879de69b94084f1fb8'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.507549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:57.521679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1208/2439] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:57 2022] DELETE /identity/v3/users/b5b22eed68ce4d879de69b94084f1fb8 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:57.874913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:57.889885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f2496-9c85-496e-87d0-02188ea14148 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:57.986151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:57.986970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:57.987467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:57.987908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] PATH_INFO: `/v3/projects/a747512840e14dfca8bf507e70623fa5` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:57.989343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a747512840e14dfca8bf507e70623fa5)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:57.991321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.065737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a747512840e14dfca8bf507e70623fa5', 'request_id': 'req-d1537e30-c5f4-4498-a436-23e918de3fe1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.073394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a747512840e14dfca8bf507e70623fa5', 'request_id': 'req-d1537e30-c5f4-4498-a436-23e918de3fe1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.085171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a747512840e14dfca8bf507e70623fa5', 'request_id': 'req-d1537e30-c5f4-4498-a436-23e918de3fe1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.111385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] The token cache is being invalidate because project a747512840e14dfca8bf507e70623fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:01:58.111714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.112514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-d1537e30-c5f4-4498-a436-23e918de3fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.119663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1232/2440] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:01:57 2022] DELETE /identity/v3/projects/a747512840e14dfca8bf507e70623fa5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:58.126133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.128339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.128958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.174698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.176254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.253755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1209/2441] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:58.261873 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.262191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.262459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.284945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.286384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.294538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1233/2442] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:58.298992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.308794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c2a83-1744-4606-92bb-4da3d9ed117d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.313977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.314658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.314941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.315204 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.315923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:01:58.316854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.318076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.330302 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1210/2443] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:01:58.335359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.345168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a9d9f-efa8-4ccd-a41f-6d232a0d92e7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.350220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.350913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.351197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.351476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.352129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.354510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.389163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1234/2444] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:01:58.393460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.405146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea0ac-5cbf-42a2-9f31-fb8e7eb807df admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.492907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.493737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.494201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.494780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.495557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.498863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.528168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1211/2445] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:01:58.532286 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.545542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f622-10a1-417c-9fce-3418e923f9c4 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.552233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.553542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.553987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.554598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.555516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.558899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.579416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1235/2446] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:58.585009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.595130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4aeb-8b5f-453b-b002-fe9c08897c27 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.600010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.600708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.601765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.601765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.602120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.603824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e3a53b-6abe-4bba-8f56-50a736393610 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.615510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1212/2447] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:58.620092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.630886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6272e-38a6-4b4b-ad9e-13c785777ebe admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.635892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.636576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.636855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.637105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] PATH_INFO: `/v3/projects/12334580a162423aaf7e5ea84cd0d590/users/624d22e884154931a198491481459d93/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.655821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=12334580a162423aaf7e5ea84cd0d590, user_id=624d22e884154931a198491481459d93, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.659132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.674504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1236/2448] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:01:58 2022] PUT /identity/v3/projects/12334580a162423aaf7e5ea84cd0d590/users/624d22e884154931a198491481459d93/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:58.680224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec519-530e-4409-b60b-9d2817b8b810 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.680590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec519-530e-4409-b60b-9d2817b8b810 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.680925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ec519-530e-4409-b60b-9d2817b8b810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.702061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b08ec519-530e-4409-b60b-9d2817b8b810 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.703524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b08ec519-530e-4409-b60b-9d2817b8b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.773759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1213/2449] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:01:58.784106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.786293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.808975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:58.873141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978e4ff-8345-40ca-b7c9-4e09fdf420cc admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.877456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.878208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.878574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.878846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.889280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.890856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.902084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1237/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:59.185572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c911-a8b7-400e-aeff-0337d659a6f9 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:59.186056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c911-a8b7-400e-aeff-0337d659a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:59.186436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c911-a8b7-400e-aeff-0337d659a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:59.222241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:59.229792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-81f5c911-a8b7-400e-aeff-0337d659a6f9 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:59.231756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-81f5c911-a8b7-400e-aeff-0337d659a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:59.323895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1214/2451] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:59.346786 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:59.348710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:59.494108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14a408-57ea-4629-ba5a-693ba91bbfb9 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:59.504318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:59.505018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:59.505452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:59.505728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:59.515438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:59.517789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:59.529077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1238/2452] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:01:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:02:00.675861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeb310-bd14-4e07-95c6-fda607e7b2c3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.676417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeb310-bd14-4e07-95c6-fda607e7b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.676417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeb310-bd14-4e07-95c6-fda607e7b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.733200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4efeb310-bd14-4e07-95c6-fda607e7b2c3 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:00.734514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4efeb310-bd14-4e07-95c6-fda607e7b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:00.749180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1215/2453] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:00.791863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.812775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cfec-bf77-427b-a42d-65cc872c4363 service swift] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.831152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.832238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.833091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.833574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.855530 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.860305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.892427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1239/2454] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:02:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:02:08.232379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670246f3-0e49-4e78-a507-670524ed213e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.233469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670246f3-0e49-4e78-a507-670524ed213e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.234027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670246f3-0e49-4e78-a507-670524ed213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.300786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-670246f3-0e49-4e78-a507-670524ed213e None None] MFA Rules not processed for user `624d22e884154931a198491481459d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:08.303124 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-670246f3-0e49-4e78-a507-670524ed213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:08.445234 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.446130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1216/2455] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:08.461383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2070f-83d4-4441-acbe-a95e7452e0fc service swift] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.468868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.469883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.470431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.470879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.471838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.474466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.554687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1240/2456] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:02:08 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:08.558706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670246f3-0e49-4e78-a507-670524ed213e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.576541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670246f3-0e49-4e78-a507-670524ed213e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.775317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.776104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.776608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.780780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.781915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.791533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.864224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1217/2457] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:02:08 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:08.865495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.880701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d9942-ecc4-4035-919a-9cee0860a94b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.886747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.887945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.888432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.888863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.889701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.891913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.907183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1241/2458] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:02:08.911122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.935274 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cafc0f-094e-4e09-b1a0-f46abc12b5b7 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.943909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.945150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.945964 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.946659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] PATH_INFO: `/v3/projects/5e82d57a52324e19a365da299e0f49a0/users/823280b1838242d5aa35ab7ca7eae4d4/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:09.009979 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e82d57a52324e19a365da299e0f49a0, user_id=823280b1838242d5aa35ab7ca7eae4d4, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:09.013367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:09.036528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:09.052349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1218/2459] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:08 2022] PUT /identity/v3/projects/5e82d57a52324e19a365da299e0f49a0/users/823280b1838242d5aa35ab7ca7eae4d4/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:09.053989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:09.090864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe5816-92f0-4e61-bed8-4d18ff9e19a3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:09.273528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:09.274827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:09.275568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:09.275890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:09.277201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:09.286445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1cc49-59e9-4d9f-a412-29b60ebde70f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:09.298491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:09.325342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:09.326350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1242/2460] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:09.340899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196d5cc-f78f-427a-b9c7-f160b301d3b6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:09.347747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:09.348798 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:09.349468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:09.349933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] PATH_INFO: `/v3/projects/5e82d57a52324e19a365da299e0f49a0/users/823280b1838242d5aa35ab7ca7eae4d4/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:09.358087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e82d57a52324e19a365da299e0f49a0, user_id=823280b1838242d5aa35ab7ca7eae4d4, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:09.361158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:09.402751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1219/2461] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:09 2022] PUT /identity/v3/projects/5e82d57a52324e19a365da299e0f49a0/users/823280b1838242d5aa35ab7ca7eae4d4/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:02:19.315315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:19.392306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=5,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:21.840637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65389-96ba-4f70-bd73-1255053b558e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:21.841349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65389-96ba-4f70-bd73-1255053b558e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:21.841349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a65389-96ba-4f70-bd73-1255053b558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:21.927304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b3a65389-96ba-4f70-bd73-1255053b558e None None] MFA Rules not processed for user `823280b1838242d5aa35ab7ca7eae4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:21.928781 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b3a65389-96ba-4f70-bd73-1255053b558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:22.133355 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1243/2462] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:02:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:22.141324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:22.146753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:22.331160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50027697-01b4-4d57-b28f-5612729c2ec0 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:22.341055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:22.342126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:22.342687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:22.343134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:22.505000 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:22.510167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:22.548943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1220/2463] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:24.784390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185774-5126-4687-a270-75aa7087941b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:24.785029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185774-5126-4687-a270-75aa7087941b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:24.785819 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185774-5126-4687-a270-75aa7087941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:24.811076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b9185774-5126-4687-a270-75aa7087941b None None] MFA Rules not processed for user `624d22e884154931a198491481459d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:24.812317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b9185774-5126-4687-a270-75aa7087941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:24.823319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1244/2464] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:02:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:32.522264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=56 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:34.803098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=37 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:12.228007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:12.296767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccc8dc-4425-440d-8ef6-954de7dd328c service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:12.300861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:12.301477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:12.301746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:12.302029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:12.303981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:12.325735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:12.327247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:12.339691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1221/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:22.334084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=26 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:22.933016 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185774-5126-4687-a270-75aa7087941b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:22.934642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b9185774-5126-4687-a270-75aa7087941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:23.011380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185774-5126-4687-a270-75aa7087941b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:23.015118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:23.015741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:23.016020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:23.016290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:23.018171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:23.032397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:23.033841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0a6c4-3e85-46e5-b5d5-ffe6ba442894 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:23.046516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1245/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:26.803317 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:26.804819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:26.816800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8da170-75e4-4ded-8287-59b29fa80b3c service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:26.821081 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:26.821801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:26.822102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:26.822414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:26.824719 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:26.893220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:26.894736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:26.905006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1222/2467] 10.222.0.52 () {60 vars in 1396 bytes} [Tue Sep 13 20:03:26 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:28.244554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:28.245073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:28.245270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:28.265687 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:28.267244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:28.329756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1246/2468] 10.222.0.52 () {58 vars in 967 bytes} [Tue Sep 13 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:28.375176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:28.446064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cdab9-168a-49c5-8543-29db3b145cbe service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:28.451668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:28.452265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:28.452881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:28.453147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:28.462455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:28.464057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:28.473080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1223/2469] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:03:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:03:28.489706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:28.500463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e6d5d-61d7-49e2-af32-d3e94020754f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:28.505995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:28.507094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:28.507430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:28.507940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:28.510434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:28.587085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:28.588668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:28.599295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1247/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:38.469337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=60 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:38.594820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=79 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:11.230177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:11.240168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a8b08-050c-4398-94da-984714870eb0 service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:11.244675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:11.245294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:11.245572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:11.245840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:11.247929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:11.310568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:11.312129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:11.322551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1224/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:21.317172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=28 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:25.337167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:25.351525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ee784-ad73-415b-aaf3-5976357a3578 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:25.356341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:25.356995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:25.357281 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:25.357548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:25.359746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:25.372721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:25.374210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:25.383834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1248/2472] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:35.379458 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:56.561493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:56.572963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2709a5b-c3af-4e2c-8a08-62a2579ac964 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:56.577533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:56.578163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:56.578557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:56.578829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:56.580848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:56.641494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:56.642988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:56.652117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1225/2473] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:06.647841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=28 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:06:37.913492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:37.915465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:37.989042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919fb9-c72a-452d-97f0-509bad9d53a2 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:37.992918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:37.993528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:37.993801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:37.994065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:37.996009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:38.090564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:38.092033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:38.101448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1249/2474] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:48.097046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=56 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:19.497835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:19.499649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:19.641110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d447b71-4ae0-4631-bb47-c78dab7cc8ca service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:19.649735 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:19.650847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:19.651194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:19.651512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:19.653514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:19.667246 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:19.668795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:19.679114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1226/2475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:21.681612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:21.693560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312157cd-bb13-4185-86c8-8fa3840ce5d0 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:21.697479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:21.698085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:21.698395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:21.698670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:21.708830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:21.710245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:21.719448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1250/2476] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:29.674642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=59 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:31.715363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:25.969427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:25.990287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0b4d-0658-45cb-9ff0-7a25da0c10df service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:25.994532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:25.995156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:25.995432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:25.995701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:26.084644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:26.086170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:26.113618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1227/2477] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:27.266615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:27.268268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:27.317108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c18544-6310-42fb-a7a6-1616ec4b17e9 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:27.323908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:27.324457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:27.324738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:27.325223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:27.334431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:27.336177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:27.365301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1251/2478] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:27.823084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:27.834547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc7b64-e048-4686-af8d-5e05fb5c1c34 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:27.839892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:27.840545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:27.843946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:27.844355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:27.855744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:27.857525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:27.889336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1228/2479] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:29.857420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:29.870822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233019d-5538-4043-8f17-3676a4efdd40 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:29.876501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:29.877158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:29.877616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:29.878135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:29.881023 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:29.893642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:29.895307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:29.907243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1252/2480] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:33.315980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:33.339352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d319c-a194-444d-9330-632923aa3832 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:33.344413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:33.345031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:33.345311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:33.345584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:33.347423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:33.359603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:33.361636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:33.375693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1229/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:33.379721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:33.392115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0e974-36fb-46f7-8292-9ebc7e5c63c8 service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:33.397115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:33.397876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:33.398226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:33.398536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] PATH_INFO: `/v3/users/624d22e884154931a198491481459d93` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:33.399704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624d22e884154931a198491481459d93)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:33.401603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:33.450785 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624d22e884154931a198491481459d93'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.459206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624d22e884154931a198491481459d93'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.472003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624d22e884154931a198491481459d93'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.472749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624d22e884154931a198491481459d93'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.480770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624d22e884154931a198491481459d93'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.486871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1253/2482] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:08:33 2022] DELETE /identity/v3/users/624d22e884154931a198491481459d93 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:08:33.669635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:33.682070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9ce8d-689d-413c-af5c-7b08c2ac3884 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:33.742212 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:33.743036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:33.743318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:33.743599 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] PATH_INFO: `/v3/projects/12334580a162423aaf7e5ea84cd0d590` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:33.744731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12334580a162423aaf7e5ea84cd0d590)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:33.746686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:33.779359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:33.795304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12334580a162423aaf7e5ea84cd0d590', 'request_id': 'req-93de26e9-2453-4660-a9a1-8ba84ff03ac2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.800660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12334580a162423aaf7e5ea84cd0d590', 'request_id': 'req-93de26e9-2453-4660-a9a1-8ba84ff03ac2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.806973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12334580a162423aaf7e5ea84cd0d590', 'request_id': 'req-93de26e9-2453-4660-a9a1-8ba84ff03ac2'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.834100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] The token cache is being invalidate because project 12334580a162423aaf7e5ea84cd0d590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:08:33.834351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.835198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.836886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1230/2483] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:08:33 2022] DELETE /identity/v3/projects/12334580a162423aaf7e5ea84cd0d590 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:08:33.841695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:33.843929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:33.858573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22aae2-dee5-4f21-ad68-8a801b805199 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:33.916932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:33.917565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:33.917874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:33.918143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] PATH_INFO: `/v3/users/823280b1838242d5aa35ab7ca7eae4d4` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:33.919304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=823280b1838242d5aa35ab7ca7eae4d4)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:33.921246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:33.925843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:33.974603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '823280b1838242d5aa35ab7ca7eae4d4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.981984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '823280b1838242d5aa35ab7ca7eae4d4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.994158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '823280b1838242d5aa35ab7ca7eae4d4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:33.994861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '823280b1838242d5aa35ab7ca7eae4d4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.002393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '823280b1838242d5aa35ab7ca7eae4d4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.009721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1254/2484] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:08:33 2022] DELETE /identity/v3/users/823280b1838242d5aa35ab7ca7eae4d4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:08:34.212109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:34.213655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:34.227178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de26e9-2453-4660-a9a1-8ba84ff03ac2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:34.292186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:34.292844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:34.293122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:34.293393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] PATH_INFO: `/v3/projects/5e82d57a52324e19a365da299e0f49a0` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:34.294769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e82d57a52324e19a365da299e0f49a0)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:34.297080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:34.347167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e82d57a52324e19a365da299e0f49a0', 'request_id': 'req-e21d8b51-a47e-472a-b298-1d007921e337'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.353196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e82d57a52324e19a365da299e0f49a0', 'request_id': 'req-e21d8b51-a47e-472a-b298-1d007921e337'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.359648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e82d57a52324e19a365da299e0f49a0', 'request_id': 'req-e21d8b51-a47e-472a-b298-1d007921e337'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.373448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] The token cache is being invalidate because project 5e82d57a52324e19a365da299e0f49a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:08:34.373739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.374460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:08:34.376154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1231/2485] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:08:34 2022] DELETE /identity/v3/projects/5e82d57a52324e19a365da299e0f49a0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:08:34.427493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:34.429082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:34.493160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675261d3-8f50-49b5-bdf5-c15e90868364 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:34.555091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:34.555847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:34.556122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:34.556393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:34.558586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:34.619290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:34.641790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:34.643283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:34.654307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1255/2486] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:44.370255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=70,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:44.649308 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=9 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:11.014009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:11.016006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:11.029746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d8b51-a47e-472a-b298-1d007921e337 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:11.034165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:11.034799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:11.035087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:11.035359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:11.037411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:11.123739 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:11.125905 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:11.140540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1232/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:21.133408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=37 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:23.531443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:23.533103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:23.608864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d016-c76c-4038-aa3d-334860017ec5 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:23.613271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:23.614142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:23.614614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:23.614898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:23.617373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:23.632193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:23.633646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:23.645318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1256/2488] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:09:27.180801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:27.191319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5ea57-9574-47d6-9869-106e914d3ba2 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:27.195839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:27.196464 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:27.196734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:27.196999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:27.199424 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:27.224830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:27.224830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:27.224830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1233/2489] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:33.639956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=31 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:37.226727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:56.931289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:56.933030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:56.948285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6250-3202-440d-b667-057766382c55 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:56.959096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:56.959874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:56.960159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:56.960433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:56.963354 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:57.050821 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:57.052473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:57.072167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1257/2490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:10:07.069512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=41 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:37.204804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:37.206498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:37.218295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d598-52ab-4dcc-b577-19564e3ab3cc service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:37.223521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:37.224403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:37.224774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:37.225201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:37.227944 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:37.311360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:37.312804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:37.331253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1234/2491] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:47.328356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=41 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:12:21.543176 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:12:21.544595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:12:21.615383 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b509dc5-bc96-4e6b-a848-d266cb52255c service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:12:21.621666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:12:21.622254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:12:21.622553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:12:21.622815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:12:21.624949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:12:21.636783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:12:21.638569 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:12:21.649210 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1258/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:12:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:12:31.644560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=32 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:27.373733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:27.383046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ad89d-2dde-45ec-9131-e7075fa0b7e7 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:27.387450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:27.388068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:27.388337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:27.388603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:27.390655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:27.461913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:27.463390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:27.472548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1235/2493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:28.446156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:28.459137 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d39f42-2b00-44a4-9f12-ce9c29137062 service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:28.466330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:28.466965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:28.467239 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:28.467503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:28.469486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:28.545914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:28.547620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:28.557730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1259/2494] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:34.556258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:34.557918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:34.571187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19687de7-c56b-4f97-9b9f-b0e736d056a1 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:34.575116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:34.575672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:34.575943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:34.576214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:34.588682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:34.590128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:34.601847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1236/2495] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:38.553024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=33 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:44.596530 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=39 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:11.965160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:11.974648 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781cd5c-5e56-4db9-b1f9-8279e5e83ea0 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:11.978585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:11.979189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:11.979468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:11.979736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:11.988709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:11.990213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:12.010527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1260/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:22.007652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=10 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:30.188628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.197966 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b8012-eaf0-4f22-afc4-f8b24bd41ee3 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.202335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.202975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.203251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.203518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.211010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.212818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:30.220534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1237/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:36.774061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:36.784971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6aee3-65ed-4858-9716-dac4ef7d1912 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:36.789674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:36.790272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:36.790581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:36.790845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:36.802516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:36.803949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:36.815098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1261/2498] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:40.217174 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:46.810306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:57.583242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:57.585525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:57.606410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f1465-cd1d-4552-9243-25991ecce16f service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:57.615226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:57.615834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:57.616110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:57.616378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:57.624657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:57.626265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:57.635779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1238/2499] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:07.631304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:25.571594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:25.583410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865c09c-5ab4-420e-97ec-1caab5243d58 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:25.589508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:25.590208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:25.590514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:25.590818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:25.592564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:25.605561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:25.607433 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d47e1-7739-4557-8949-2ab1df287b3a service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:25.619867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1262/2500] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:26.478467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:26.513902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef964-26fc-43e1-b09b-39715f81b264 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:26.522923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:26.525774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:26.526143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:26.526882 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:26.529505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:26.549292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:26.550971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:26.564463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1239/2501] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:29.912868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:29.913838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:29.914112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:29.973424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:15:29.974685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:30.052486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1263/2502] 10.222.0.52 () {58 vars in 967 bytes} [Tue Sep 13 20:15:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:15:30.103572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:30.105227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:30.323358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea30c5c-8982-48fa-8c9f-93fb50179bae service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:30.328825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:30.329469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:30.329744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:30.330013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:30.338763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:30.340438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:30.349986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1240/2503] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:15:30.846514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:30.856509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818a383-da11-4b85-b1f3-1389b7c07176 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:30.860844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:30.861478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:30.861755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:30.862022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:30.863966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:30.929767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:30.931244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:30.940815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1264/2504] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:34.577958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:34.579570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:34.594980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f23ac-0584-4389-98b8-b7af768aece3 service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:34.600584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:34.601265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:34.601634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:34.601983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:34.603893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:34.616898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:34.618509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:34.630550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1241/2505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:37.934357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:37.946019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61e346-f062-4b87-b9ae-5f578d32f676 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:37.951861 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:37.952470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:37.952790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:37.953034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:37.955103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:37.968211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:37.970051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:37.981334 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1265/2506] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:37.997389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:37.998913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.013566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe6f3f-e261-41d1-bee9-e731bc2717b6 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:38.018836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:38.019528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.019950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.020067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] PATH_INFO: `/v3/users/66b7e2cda35c473597343822e622a3bd` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.021151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b7e2cda35c473597343822e622a3bd)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:38.023264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:38.070224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b7e2cda35c473597343822e622a3bd'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.079684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b7e2cda35c473597343822e622a3bd'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.092556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b7e2cda35c473597343822e622a3bd'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.093287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b7e2cda35c473597343822e622a3bd'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.100334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b7e2cda35c473597343822e622a3bd'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.106330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1242/2507] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:15:37 2022] DELETE /identity/v3/users/66b7e2cda35c473597343822e622a3bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:15:38.288396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:38.299407 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2a3a4-96b0-4d7c-9e6c-d798c6be2c5f service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:38.332547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:38.362165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:38.362763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.363044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.363317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] PATH_INFO: `/v3/projects/c401b957669e43bdabf08fa232b461be` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.364858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c401b957669e43bdabf08fa232b461be)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:38.366813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:38.416815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c401b957669e43bdabf08fa232b461be', 'request_id': 'req-8771c37c-eebf-4674-ba3f-c023fa379717'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.422716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c401b957669e43bdabf08fa232b461be', 'request_id': 'req-8771c37c-eebf-4674-ba3f-c023fa379717'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.429069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c401b957669e43bdabf08fa232b461be', 'request_id': 'req-8771c37c-eebf-4674-ba3f-c023fa379717'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.442294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] The token cache is being invalidate because project c401b957669e43bdabf08fa232b461be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:15:38.442637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.443343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.444938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1266/2508] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:15:38 2022] DELETE /identity/v3/projects/c401b957669e43bdabf08fa232b461be => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:15:38.449112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:38.450869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.464894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3966fae-7731-4218-9720-496a62043e5a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:38.478884 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:38.527632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:38.528271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.528549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.528822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] PATH_INFO: `/v3/users/a7de0bda37784bb9a6717ff648317f5d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.543114 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7de0bda37784bb9a6717ff648317f5d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:38.545072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:38.596272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7de0bda37784bb9a6717ff648317f5d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.604692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7de0bda37784bb9a6717ff648317f5d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.617707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7de0bda37784bb9a6717ff648317f5d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.618272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7de0bda37784bb9a6717ff648317f5d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.625390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7de0bda37784bb9a6717ff648317f5d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.631810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1243/2509] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:15:38 2022] DELETE /identity/v3/users/a7de0bda37784bb9a6717ff648317f5d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:15:38.827014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:38.828638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.842602 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771c37c-eebf-4674-ba3f-c023fa379717 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:38.903446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:38.904094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.904374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.904643 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] PATH_INFO: `/v3/projects/2d0806d18e154bf99b8672d168247c96` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.915803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0806d18e154bf99b8672d168247c96)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:38.917729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:38.965411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0806d18e154bf99b8672d168247c96', 'request_id': 'req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.969357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:38.972077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0806d18e154bf99b8672d168247c96', 'request_id': 'req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.980171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0806d18e154bf99b8672d168247c96', 'request_id': 'req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.994262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] The token cache is being invalidate because project 2d0806d18e154bf99b8672d168247c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:15:38.994582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.995199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:15:38.996900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1267/2510] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:15:38 2022] DELETE /identity/v3/projects/2d0806d18e154bf99b8672d168247c96 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:15:48.628010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:48.991331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=2,keystone:SELECT=5,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:26.447495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:26.449277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:26.597382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bd59e-cd35-4c63-b6e7-602d7279fcfd admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:26.656008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:26.657226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:26.657506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:26.657774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:26.659514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:26.670898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:26.672532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:26.682141 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1244/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:36.678277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:27.589168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:27.590679 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:27.733076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30eb2-eda7-40ad-8fe9-1e4f0c5e8a44 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:27.739784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:27.740633 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:27.740924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:27.741187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:27.742968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:27.817340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:27.818994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:27.838484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1268/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:28.761224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:28.762876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:28.845558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f02af0-ca95-45df-ac22-3fb315a333c5 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:28.849299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:28.850077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:28.850343 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:28.850673 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:28.852544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:28.923828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:28.925442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:28.944537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1245/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:34.767064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:34.769207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:34.808772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:34.845614 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf92f4-f5b6-41df-9a45-dfe1a481f1a8 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:34.849666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:34.850265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:34.850571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:34.850837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:34.852770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:34.866506 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:34.867933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:34.878988 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1269/2514] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:38.942032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=65 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:44.873933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=17 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:11.770540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:11.772368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:11.792549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e9ead-31f5-4437-98af-45b8d5f65129 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:11.801138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:11.801998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:11.802609 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:11.802920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:11.805975 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:11.910476 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:11.912082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:11.921823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1246/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:21.917255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=37 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:37.181802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:37.183368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:37.208066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f10b4f-647a-4d9e-bfe9-a43421e27a5a service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:37.211893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:37.212508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:37.212790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:37.213072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:37.214936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:37.224456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:37.225919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:37.233727 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1270/2516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:37.433380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:37.444535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a4e48-9eb5-45e1-b340-201787ab340b service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:37.448471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:37.449097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:37.449361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:37.449637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:37.451512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:37.465631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:37.467119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:37.478307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1247/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:47.230543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=11 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:47.473439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:58.438598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:58.440336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:58.452714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c73e96-9a8c-4923-8117-73ba7b5bf9f3 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:58.457795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:58.458411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:58.458723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:58.458993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:58.461048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:58.546439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:58.548336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:58.573478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1271/2518] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:20:08.570138 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=41 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:21:11.073743 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:21:11.089184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edddf3b-968d-439f-86d0-ced2411d5b7a service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:21:11.095885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:21:11.096522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:21:11.096802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:21:11.097071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:21:11.099038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:21:11.114015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:21:11.115883 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e37a46-835b-458e-8504-5ced3afd830e service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:21:11.154127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1248/2519] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:21:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:21:21.136487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:09.481931 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:09.502817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fbb05-ade6-451f-b629-f64d065f7e7e service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:09.507845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:09.508473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:09.508748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:09.509452 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:09.511092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:09.524255 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:09.525846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:09.543672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1272/2520] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:11.023339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:11.023898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:11.023898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:11.051841 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] MFA Rules not processed for user `b07b8b251aaa456c9916c41f9a0b77e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:22:11.053029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:11.060237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1249/2521] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:22:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:22:11.376020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:11.390756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a58193-dfcc-4688-bf01-f0f840d0df60 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:11.395439 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:11.396019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:11.396303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:11.396574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:11.398701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:11.408716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:11.410198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:11.419259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1273/2522] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:11.759292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:11.760980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:11.779406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49119330-ab00-4c9a-a6f2-de4bcd2aac86 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:11.784974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:11.786493 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:11.787032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:11.787506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:11.790239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:11.803496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:11.805412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:11.816147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1250/2523] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:18.829165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:18.844598 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d5cca-7969-43c2-9588-cb14505eefd9 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:18.850928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:18.851559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:18.852038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:18.852315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:18.854305 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:18.867554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:18.869206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:18.880472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1274/2524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:21.811620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=12 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:28.875637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=18 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:29.475233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:29.476855 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:29.490278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a9dfb-be4f-4ab5-bf82-cb739de1ef8e service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:29.496406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:29.496999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:29.497270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:29.497532 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:29.506207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:29.507848 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:29.517469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1251/2525] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:39.513630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:35.946758 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:35.959390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c61c6-ec25-49a2-8651-ae95aee29277 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:35.963638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:35.964229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:35.964507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:35.964776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:35.976516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:35.977958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:35.989320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1275/2526] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:23:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:23:45.984348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:37.438575 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:37.449213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c8a6c-b548-495c-88ea-ecaba76e0584 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:37.453534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:37.454132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:37.454412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:37.454697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:37.465608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:37.467075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:37.478314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1252/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:40.169820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:40.179093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2a2c-4cee-49dd-98a0-12ccefd0650e service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:40.183567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:40.184134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:40.184446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:40.184683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:40.192545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:40.193979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:40.202700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1276/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:47.473479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:50.199064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:58.624674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:58.637414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d454d-0d1c-4ce5-a7d6-dd350b27fe7f service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:58.644439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:58.645043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:58.645311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:58.645701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:58.653987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:58.655593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:58.664577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1253/2529] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:08.660574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:11.598918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:11.610719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956462-64f6-4d2f-a2f6-32f559093fc4 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:11.615722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:11.616566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:11.616849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:11.617117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:11.625809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:11.627272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:11.636319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1277/2530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:11.679809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:11.690152 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d0623-05b6-4782-b59f-c374f97e7227 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:11.694862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:11.695468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:11.695744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:11.696040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:11.706300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:11.707754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4a12c-d17f-4d6d-a3e7-6441c465585b service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:11.717405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1254/2531] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:11.761561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:11.770339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df547570-031e-4e08-82ae-89a95cde1822 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:11.774651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:11.775251 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:11.775526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:11.775799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:11.785386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:11.786891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:11.806468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1278/2532] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:21.712835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:21.803573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=16 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:12.735416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:12.735921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:12.736010 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:12.764291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] MFA Rules not processed for user `87c4f9c4b9334b66ac89e01a278436bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:12.765605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:12.773037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1255/2533] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:27:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:13.099459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:13.114257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf1a53-7efb-4467-9538-b159dae49b49 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:13.121046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:13.121974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:13.122413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:13.122846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:13.133019 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:13.134503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:13.144340 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1279/2534] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:13.485709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:13.497768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113e692-e352-4926-b7a4-a32401d814cc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:13.503113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611427-120a-4186-b996-f0829b593b74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:13.503733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611427-120a-4186-b996-f0829b593b74 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:13.504005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611427-120a-4186-b996-f0829b593b74 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:13.504368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611427-120a-4186-b996-f0829b593b74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:13.513536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33611427-120a-4186-b996-f0829b593b74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:13.515631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33611427-120a-4186-b996-f0829b593b74 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:13.526166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1256/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:16.105656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:16.121611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50753ab4-8fa0-4662-8031-c841e7aac9dc service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:16.128397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:16.129065 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:16.129385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:16.129825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:16.139185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:16.140796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:16.151574 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1280/2536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:22.353381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611427-120a-4186-b996-f0829b593b74 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:22.368973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611427-120a-4186-b996-f0829b593b74 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:22.374838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:22.375447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.375725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.375996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.385385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:22.387026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:22.397802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1257/2537] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:25.354413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:25.365729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cf9a-f3df-4767-a81e-f8fcf28f46fa service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:25.372692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:25.373329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:25.373603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:25.373864 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:25.383707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:25.385335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:25.396637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1281/2538] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:32.393248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=18 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:33.468361 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:33.563560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98263892-22c9-4007-9572-73b802d2a037 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:33.569969 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:33.570625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:33.570907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:33.571187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:33.573692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:33.607756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:33.609354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:33.620381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1258/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:35.391884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=18 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:43.616938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=52 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:02.771863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:02.773469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:02.851202 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c7f33-1ef0-4018-96a0-12fd2e741e8f service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:02.855802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:02.856418 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:02.856694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:02.856983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:02.859811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:02.983342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:02.984797 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:03.005594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1282/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:29:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:29:13.003066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=84 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:32.145959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:32.160039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b192-b5bc-49f8-8e83-ec3a393b5e2e service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:32.164105 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:32.164712 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:32.164989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:32.165253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:32.167430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:32.257626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:32.259113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:32.278557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1259/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:29:35.319851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:35.331704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0eb5-da85-40d4-8745-e2a41e1669df service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:35.336678 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:35.337419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:35.337701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:35.337971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:35.339739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:35.352946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:35.355634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:35.366272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1283/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:29:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:29:35.385594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:35.387864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:35.401732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f301d37-1247-40f4-acf4-7e214f3ae837 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:35.406994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:35.407646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:35.407919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:35.408186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] PATH_INFO: `/v3/users/2577fc11946c4597857187ae7b464fe7` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:35.425950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2577fc11946c4597857187ae7b464fe7)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:35.427936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:35.474145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2577fc11946c4597857187ae7b464fe7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.482601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2577fc11946c4597857187ae7b464fe7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.495178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2577fc11946c4597857187ae7b464fe7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.495858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2577fc11946c4597857187ae7b464fe7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.502846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-38debb80-a4c4-4fa3-b9d9-48010503ea1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2577fc11946c4597857187ae7b464fe7'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.509046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1260/2543] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:29:35 2022] DELETE /identity/v3/users/2577fc11946c4597857187ae7b464fe7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:29:35.684718 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:35.702062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44718-6d30-42c8-918f-1391d817cd8d service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:35.760412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:35.761021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:35.761302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:35.761568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] PATH_INFO: `/v3/projects/23873d995d564c0388a4649e57821c69` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:35.772126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23873d995d564c0388a4649e57821c69)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:35.774106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:35.822508 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23873d995d564c0388a4649e57821c69', 'request_id': 'req-f4948e66-d4bc-4539-972b-2643df918b26'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.828329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23873d995d564c0388a4649e57821c69', 'request_id': 'req-f4948e66-d4bc-4539-972b-2643df918b26'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.834749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23873d995d564c0388a4649e57821c69', 'request_id': 'req-f4948e66-d4bc-4539-972b-2643df918b26'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.850206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] The token cache is being invalidate because project 23873d995d564c0388a4649e57821c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:29:35.850544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.851188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f4948e66-d4bc-4539-972b-2643df918b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:29:35.852724 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1284/2544] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:29:35 2022] DELETE /identity/v3/projects/23873d995d564c0388a4649e57821c69 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:29:35.864539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:35.864913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:35.865256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:35.895579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:35.926995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:29:35.928281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:35.989469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1261/2545] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:29:35.997604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:35.997895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:35.998170 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.018691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:29:36.019820 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:36.029105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1285/2546] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:29:36.033509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.043671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d32f2-527d-4b1e-aa45-d4ce4532f156 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.048926 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.049528 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.049802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.050073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.050752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:29:36.051049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.052634 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.065013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1262/2547] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:29:36.069628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.079325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ce435-b7f9-4d1b-9a05-5a38dd650bc6 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.084086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.084693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.084970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.085240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.085935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.087899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.128108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1286/2548] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:29:36.129370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.140729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac1814-196c-44f6-ab65-9bfd49cadf7f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.198077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.198713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.198972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.199241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.199875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.201780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.224795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1263/2549] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:29:36.229046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.239037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87bfb7-f33a-40c4-8e82-d4bc426bf587 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.243554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.244150 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.244426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.244695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.245327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.246955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.257115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1287/2550] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:29:36.261069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.270681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876039be-942d-456e-9d31-82d5766b9d89 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.275529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.276120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.276398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.276676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.277292 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.278894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456e3fc-5f0a-436e-a366-53d20cd52c47 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.289480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1264/2551] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:29:36.293495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.303477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456708e-e997-40b8-a58f-4a1067a8d571 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.309243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.309919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.310195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.310507 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] PATH_INFO: `/v3/projects/0ce046dd76e44880a92ce4f333f42b6c/users/caa989a4f21346dfa42501abc83030a1/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.326336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce046dd76e44880a92ce4f333f42b6c, user_id=caa989a4f21346dfa42501abc83030a1, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.329030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.332149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:36.346788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1288/2552] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:29:36 2022] PUT /identity/v3/projects/0ce046dd76e44880a92ce4f333f42b6c/users/caa989a4f21346dfa42501abc83030a1/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:29:36.358503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.358786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.359052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.388533 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] MFA Rules not processed for user `caa989a4f21346dfa42501abc83030a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:29:36.389781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:36.437593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:36.460158 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1265/2553] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:29:36.464765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.527824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443190c0-8659-4d6a-a4bf-b82a25c0392d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.532736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.533348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.533627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.534018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.534682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.536641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.571493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1289/2554] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:29:36.575923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.587304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a57838-5955-4de5-a54a-077aaadcdba6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.647470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.648122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.648396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.648662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.649294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.651262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.673593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1266/2555] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:29:36.677756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.688901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c838540-9025-4d90-a483-0625587ab67b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.693595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.694209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.694517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.694801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.695460 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.697099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.709542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1290/2556] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:29:36.713825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.724316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c13c4-99c1-4609-8268-a03bfb7f33df admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.729847 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.730574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.730880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.731154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] PATH_INFO: `/v3/projects/bd478b8fe2c148408999976218d31f3a/users/bb24235af97f448da12f17003959ad25/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.746518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd478b8fe2c148408999976218d31f3a, user_id=bb24235af97f448da12f17003959ad25, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.749226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.765145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1267/2557] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:29:36 2022] PUT /identity/v3/projects/bd478b8fe2c148408999976218d31f3a/users/bb24235af97f448da12f17003959ad25/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:29:36.769228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.779036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72104d-64ad-4af3-884f-8dde9e164082 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.837845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.838499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.838779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.839051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.839696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.841320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd995b-aa47-4e19-a96d-cd29f1ce6b7a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.851688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1291/2558] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:29:36.856140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:36.866764 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fb2a9-328d-4f2c-b961-ebc3332b97bb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:36.871823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:36.872521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.872788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.873062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] PATH_INFO: `/v3/projects/bd478b8fe2c148408999976218d31f3a/users/bb24235af97f448da12f17003959ad25/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.875073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd478b8fe2c148408999976218d31f3a, user_id=bb24235af97f448da12f17003959ad25, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:36.877736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:36.888803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1268/2559] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:29:36 2022] PUT /identity/v3/projects/bd478b8fe2c148408999976218d31f3a/users/bb24235af97f448da12f17003959ad25/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:29:36.898755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:36.899041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:36.899314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:36.927526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] MFA Rules not processed for user `bb24235af97f448da12f17003959ad25`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:29:36.928725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:36.981337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:37.001018 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1292/2560] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:29:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:29:46.885296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=50,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:46.997793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:12.164380 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:12.166565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:12.251370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cd1fa-814b-4356-98c9-e261b18167d6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:12.255524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:12.256131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:12.256410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:12.256681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:12.258699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:12.341801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:12.343279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:12.361529 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1269/2561] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:22.359247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=70 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:26.962169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:26.964647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:26.981264 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a7de-5369-4dc9-beb4-23e97eb24e65 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:26.985829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:26.986616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:26.986898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:26.987167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:26.989287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:27.075553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:27.077017 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:27.086296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1293/2562] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:27.171844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:27.181874 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094904b0-7985-4b61-a180-878f34609e12 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:27.186007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:27.186638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:27.186917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:27.187185 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:27.189372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:27.276611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:27.278051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:27.296375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1270/2563] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:31:27.339835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:27.348928 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884eef-e0b3-49f4-b524-7b68062e4ab5 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:27.353468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:27.354071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:27.354347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:27.354650 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:27.364532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:27.366088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:27.375683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1294/2564] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:37.293844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=41 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:37.371582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=43 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:39.436941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:39.438758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:39.523918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270590-7c42-4c31-9ac4-e7b193d63be6 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:39.529147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:39.529762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:39.530038 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:39.530310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:39.532095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:39.543368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:39.544942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:39.554774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1271/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:49.550890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=32 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:23.784128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:23.809524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:23.810983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:23.874669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579d3d-159f-40a6-9cea-f88ad7f5a9d1 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:23.880092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:23.881770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:23.882235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:23.882691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] PATH_INFO: `/v3/users/caa989a4f21346dfa42501abc83030a1` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:23.884757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa989a4f21346dfa42501abc83030a1)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:23.887552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:23.938748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046c8d1-ae2c-41ce-89be-a8b8e4377072 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:23.942720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d318e084073402e8d3385eb69c9685b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:23.943372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:23.943650 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:23.943918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:23.946206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:23.948664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa989a4f21346dfa42501abc83030a1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:23.958277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa989a4f21346dfa42501abc83030a1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:23.973736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa989a4f21346dfa42501abc83030a1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:23.974471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa989a4f21346dfa42501abc83030a1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:23.981676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-e57775a3-86a7-47db-8c25-804dd7f541aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa989a4f21346dfa42501abc83030a1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] Could not find project: 12334580a162423aaf7e5ea84cd0d590.: keystone.exception.ProjectNotFound: Could not find project: 12334580a162423aaf7e5ea84cd0d590. Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 13 20:33:23.988540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:33:23.989910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:33:23.991040 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application with Lock( Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 20:33:23.991990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 20:33:23.993062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 12334580a162423aaf7e5ea84cd0d590. Sep 13 20:33:23.993062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: ERROR keystone.server.flask.application Sep 13 20:33:23.993204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1295/2566] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:23 2022] DELETE /identity/v3/users/caa989a4f21346dfa42501abc83030a1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:23.993395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:23.993692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:23.994137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1272/2567] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:23 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 13 20:33:23.994499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:24.016941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:24.018345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:24.077431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1296/2568] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:24.088433 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:24.090235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:24.160348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb150b-9b37-42a0-b98e-094e499f59be service glance] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:24.215269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:24.215888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:24.216168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:24.216435 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:24.225763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:24.227314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:24.237817 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1273/2569] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:24.588369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:24.590071 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:24.604472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344b52d-07e8-4fea-b816-0eb236724d4c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:24.609009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:24.609651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:24.609925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:24.610191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] PATH_INFO: `/v3/projects/0ce046dd76e44880a92ce4f333f42b6c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:24.611316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce046dd76e44880a92ce4f333f42b6c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:24.613311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:24.629630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:24.670461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce046dd76e44880a92ce4f333f42b6c', 'request_id': 'req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.677421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce046dd76e44880a92ce4f333f42b6c', 'request_id': 'req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.684037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce046dd76e44880a92ce4f333f42b6c', 'request_id': 'req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.697423 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] The token cache is being invalidate because project 0ce046dd76e44880a92ce4f333f42b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:24.697722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.698385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.699952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1297/2570] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:24 2022] DELETE /identity/v3/projects/0ce046dd76e44880a92ce4f333f42b6c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:24.704899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:24.714497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592708d0-56c9-45ed-a793-79430b3d2711 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:24.747869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:24.778011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:24.778710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:24.779008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:24.779278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] PATH_INFO: `/v3/users/bb24235af97f448da12f17003959ad25` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:24.780795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb24235af97f448da12f17003959ad25)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:24.783013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:24.869920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb24235af97f448da12f17003959ad25'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.884118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb24235af97f448da12f17003959ad25'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.900619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb24235af97f448da12f17003959ad25'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.901422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb24235af97f448da12f17003959ad25'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.909066 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb24235af97f448da12f17003959ad25'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:24.916168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1274/2571] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:24 2022] DELETE /identity/v3/users/bb24235af97f448da12f17003959ad25 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:25.207587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.209313 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.230306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361a4c7-f9cf-4d39-ac6f-ce7b749de5b6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.292535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.294885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.305924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.306847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.307213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.307465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] PATH_INFO: `/v3/projects/bd478b8fe2c148408999976218d31f3a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.312630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd478b8fe2c148408999976218d31f3a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.314903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.368663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd478b8fe2c148408999976218d31f3a', 'request_id': 'req-7e42684c-0952-4a69-8c25-042b5d3aae8c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:25.376717 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd478b8fe2c148408999976218d31f3a', 'request_id': 'req-7e42684c-0952-4a69-8c25-042b5d3aae8c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:25.384097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd478b8fe2c148408999976218d31f3a', 'request_id': 'req-7e42684c-0952-4a69-8c25-042b5d3aae8c'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:25.393737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831cadf-bb97-4e75-9992-1686a823f92d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.397704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] The token cache is being invalidate because project bd478b8fe2c148408999976218d31f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:25.398076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:25.406824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7e42684c-0952-4a69-8c25-042b5d3aae8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:25.408628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1298/2572] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:25 2022] DELETE /identity/v3/projects/bd478b8fe2c148408999976218d31f3a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:25.413472 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.413822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.414157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.435842 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:25.437056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.456063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:25.475434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:25.479603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.480346 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.480759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.481069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.483308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.497334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.499012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bf95f-288b-4146-8b95-36e7e6bc5efe service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.509992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1275/2573] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:25.511014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1299/2574] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:33:25.520228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.520589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.520963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.542820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:25.544005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.552256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1276/2575] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:25.556456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.566187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3ae4a-35c8-44ef-bbb2-9ef66b600a62 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.571116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.571816 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.571980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.572247 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.572923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:33:25.573221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.574852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.586568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1300/2576] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:33:25.591014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.600415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb37ac-c84b-457c-bff9-da22cad032ea None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.605808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.606438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.606721 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.606990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.607640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.609563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.642755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1277/2577] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:25 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:25.646866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.657100 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89327e3a-f6bb-4661-bd9d-b11515768ec6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.715371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.716122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.716384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.716685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.717346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.719519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.744408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1301/2578] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:25 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:25.745715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.758229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328da7a-5a01-4983-9367-a70102b4a8bc admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.764588 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.765555 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.765815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.766067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.766748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.768339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.780623 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1278/2579] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:25.785416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.798563 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb648d-2385-4a7b-a029-b7d51ef82566 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.803299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.803924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.804207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.804482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.805288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.807107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e2249-979f-4a1c-98ad-49b184581ab3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.819531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1302/2580] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:33:25.827090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.837773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769cce0-a3f0-4846-a3d7-b0f2a159583d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:25.844492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:25.845144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.845417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.845680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] PATH_INFO: `/v3/projects/0cf9cb8a74924e77b3612f9418ad0d78/users/61f6c4b875fc41a9948dc0b2b36e0e72/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.860860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf9cb8a74924e77b3612f9418ad0d78, user_id=61f6c4b875fc41a9948dc0b2b36e0e72, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:25.863585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:25.881347 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1279/2581] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:25 2022] PUT /identity/v3/projects/0cf9cb8a74924e77b3612f9418ad0d78/users/61f6c4b875fc41a9948dc0b2b36e0e72/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:25.883157 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b0983-59f6-4206-9132-578d0b70f291 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:25.883528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b0983-59f6-4206-9132-578d0b70f291 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:25.883813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b0983-59f6-4206-9132-578d0b70f291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:25.903393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b03b0983-59f6-4206-9132-578d0b70f291 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:25.904607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b03b0983-59f6-4206-9132-578d0b70f291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:25.963716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1303/2582] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:25.973438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:25.975339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:26.034446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef122c65-b523-40ed-906d-74368e08f598 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:26.042171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:26.042812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:26.043070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:26.043355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:26.054397 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:26.055910 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:26.065674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1280/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:26.695935 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:26.696429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:26.696534 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:26.725344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] MFA Rules not processed for user `61f6c4b875fc41a9948dc0b2b36e0e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:26.726627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:26.760692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:26.797623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1304/2584] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:26.802176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:26.811838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee632d-3176-4d2a-ad9f-00eed132126d service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:26.816264 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:26.816862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:26.817138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:26.817408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:26.818037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:26.820006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:26.850309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1281/2585] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:26 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:26.854398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:26.865955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ef43-bed0-414d-9f6b-eac9fe33fed0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:26.924194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:26.924813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:26.925099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:26.925373 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:26.926051 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:26.928010 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:26.954674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1305/2586] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:26 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:33:26.959022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:26.969594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96b3c4-ae2c-42de-90e8-832ffb234619 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:26.974593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:26.975206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:26.975537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:26.975747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:26.976389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:26.977958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:26.988822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1282/2587] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:26.992982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:27.003527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ab5f3-064b-4e30-b624-ca4cdbafd1c1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:27.008116 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:27.008774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:27.009054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:27.009333 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] PATH_INFO: `/v3/projects/494a9adfea1744119f11ecb429df5590/users/f66ecd8ab2aa4bf2992701c532a63dc0/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:27.024127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494a9adfea1744119f11ecb429df5590, user_id=f66ecd8ab2aa4bf2992701c532a63dc0, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:27.026985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:27.042079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1306/2588] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:26 2022] PUT /identity/v3/projects/494a9adfea1744119f11ecb429df5590/users/f66ecd8ab2aa4bf2992701c532a63dc0/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:27.048012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:27.061268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241757e-5c54-4141-b897-22b3a74f986f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:27.105149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:27.122826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:27.123451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:27.123730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:27.124123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:27.125020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:27.126342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d3da0-33a1-4991-973d-51f1de3b2eed admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:27.136908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1283/2589] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:27.141597 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:27.152776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac41f0f-2c27-498f-84ad-5272eb400189 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:27.157512 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:27.158272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:27.158581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:27.158867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] PATH_INFO: `/v3/projects/494a9adfea1744119f11ecb429df5590/users/f66ecd8ab2aa4bf2992701c532a63dc0/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:27.160791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=494a9adfea1744119f11ecb429df5590, user_id=f66ecd8ab2aa4bf2992701c532a63dc0, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:27.163521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:27.173945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1307/2590] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:27 2022] PUT /identity/v3/projects/494a9adfea1744119f11ecb429df5590/users/f66ecd8ab2aa4bf2992701c532a63dc0/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:27.820662 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:27.820945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:27.821214 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:27.850566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] MFA Rules not processed for user `f66ecd8ab2aa4bf2992701c532a63dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:27.851727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:27.927967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1284/2591] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:27.939108 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:27.941069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:28.017695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158902-3b55-4ce0-a09a-a5faa5407ebc admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:28.021749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:28.022393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:28.022666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:28.022967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:28.055952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:28.083607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:28.085068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:28.094146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1308/2592] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:29.477288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:29.487371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dd66f-1c43-4e33-8caf-27a244303a4e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:29.492118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:29.493047 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:29.493406 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:29.493825 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:29.504126 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:29.505601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:29.515607 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1285/2593] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:30.562732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:30.623073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c173-f208-413b-b5ea-0dc1be3acff1 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:30.626990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:30.627606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:30.627885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:30.628156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:30.629998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:30.690962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:30.692401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:30.702587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1309/2594] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:31.235620 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:31.237216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:31.255096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce09d-146e-48dd-adeb-b6cd0a481ef9 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:31.259873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:31.260506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.260846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.261111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] PATH_INFO: `/v3/users/7c17f29c73f643ca83a08f4e44e6279b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.262213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c17f29c73f643ca83a08f4e44e6279b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:31.264171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:31.311091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c17f29c73f643ca83a08f4e44e6279b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.319715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c17f29c73f643ca83a08f4e44e6279b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.331959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c17f29c73f643ca83a08f4e44e6279b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.332576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c17f29c73f643ca83a08f4e44e6279b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.339669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f7245ed1-8505-44c0-9307-eb0d050cfc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c17f29c73f643ca83a08f4e44e6279b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.345661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1286/2595] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:31 2022] DELETE /identity/v3/users/7c17f29c73f643ca83a08f4e44e6279b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:31.528066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:31.542391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bab86-24fb-4504-80b3-c09bff183df6 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:31.598456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:31.599094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.599371 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.599638 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] PATH_INFO: `/v3/projects/e0a89bb1e9c24a3d8ee45633959e59ea` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.601122 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a89bb1e9c24a3d8ee45633959e59ea)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:31.603086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:31.634668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:31.652256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a89bb1e9c24a3d8ee45633959e59ea', 'request_id': 'req-542e90b4-dc51-475f-b0b8-507136214028'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.658238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a89bb1e9c24a3d8ee45633959e59ea', 'request_id': 'req-542e90b4-dc51-475f-b0b8-507136214028'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.664973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a89bb1e9c24a3d8ee45633959e59ea', 'request_id': 'req-542e90b4-dc51-475f-b0b8-507136214028'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.678855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] The token cache is being invalidate because project e0a89bb1e9c24a3d8ee45633959e59ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:31.679146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.679791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-542e90b4-dc51-475f-b0b8-507136214028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:31.681291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1310/2596] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:31 2022] DELETE /identity/v3/projects/e0a89bb1e9c24a3d8ee45633959e59ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:31.687209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.687490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.687778 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.708785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:31.709984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:31.729704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:31.771976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1287/2597] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:33:31.779929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.780219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.780495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.800410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:31.801617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:31.809801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1311/2598] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:31.814015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:31.824364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07494-90b0-4872-a2d3-02ed3178e433 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:31.829685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:31.830325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.830639 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.830897 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.831551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:33:31.831858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:31.833457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:31.845161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1288/2599] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:33:31.849388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:31.859391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b75688-aa0d-4028-a548-9ee1c3352efb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:31.864903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:31.865510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.865794 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.866063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.866736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:31.868660 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:31.902735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1312/2600] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:31 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:31.907576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:31.921606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0f37d-52ec-4a44-8836-765b7f17d424 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:31.981108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:31.981722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:31.982003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:31.982298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:31.982955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:31.984893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:32.006972 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1289/2601] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:31 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:32.011284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:32.021838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4171f4-4880-4092-afd9-4ec295ec2e5d admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:32.027586 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:32.028201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:32.028482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:32.028752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:32.029398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:32.031063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:32.041177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1313/2602] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:32.045283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:32.056119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2639fb-fd1a-419f-814e-42edc3bfff8e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:32.061765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:32.062420 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:32.062716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:32.062989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:32.063640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:32.065325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379616c2-cbc5-464a-8ece-46d2ada2927b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:32.077182 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1290/2603] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:32.081496 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:32.092083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76f6d7-f0bf-4c27-ac0a-aa41ad10c147 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:32.097030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:32.097689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:32.097965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:32.098238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] PATH_INFO: `/v3/projects/51e17aac08644c188e3fdc4e69abe464/users/25493a1eedf54816b3928c9a81860311/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:32.113237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e17aac08644c188e3fdc4e69abe464, user_id=25493a1eedf54816b3928c9a81860311, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:32.115912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:32.130154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1314/2604] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:32 2022] PUT /identity/v3/projects/51e17aac08644c188e3fdc4e69abe464/users/25493a1eedf54816b3928c9a81860311/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:32.142078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:32.142404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:32.142668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:32.172272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] MFA Rules not processed for user `25493a1eedf54816b3928c9a81860311`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:32.173506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:32.243695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1291/2605] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:32.291744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:32.293527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:32.353465 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d117a3-9205-4d1d-8c00-5aa613d8b628 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:32.357292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:32.357909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:32.358184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:32.358488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:32.368004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:32.369516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236000e2-e002-4bdf-b1f4-b168ba5cfd31 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:32.378499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1315/2606] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:40.538118 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:40.565732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:40.611307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f24bf-195f-460e-b935-ebb33fb8f93c None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:40.616058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:40.616688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:40.616963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:40.617233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] PATH_INFO: `/v3/users/25493a1eedf54816b3928c9a81860311` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:40.618539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25493a1eedf54816b3928c9a81860311)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:40.620463 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:40.669445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25493a1eedf54816b3928c9a81860311'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:40.678644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25493a1eedf54816b3928c9a81860311'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:40.690339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25493a1eedf54816b3928c9a81860311'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:40.690991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25493a1eedf54816b3928c9a81860311'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:40.698291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25493a1eedf54816b3928c9a81860311'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:40.704358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1292/2607] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:40 2022] DELETE /identity/v3/users/25493a1eedf54816b3928c9a81860311 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:40.709149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:40.709659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:40.709659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:40.733608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:40.734898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:40.800050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:40.819223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1316/2608] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:40.826543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:40.828265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:40.890353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7b7e2-9f3d-49f0-bc42-9e2d5443e3c1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:40.893981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:40.894621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:40.894903 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:40.895168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:40.904172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:40.905605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:40.915178 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1293/2609] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:33:41.213398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:41.214964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:41.226890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31197c4-8a99-4261-83f6-9d76b7a65ce3 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:41.231461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:41.232128 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:41.232399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:41.232635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] PATH_INFO: `/v3/projects/51e17aac08644c188e3fdc4e69abe464` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:41.233729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e17aac08644c188e3fdc4e69abe464)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:41.235657 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:41.285179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e17aac08644c188e3fdc4e69abe464', 'request_id': 'req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:41.292587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e17aac08644c188e3fdc4e69abe464', 'request_id': 'req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:41.300007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e17aac08644c188e3fdc4e69abe464', 'request_id': 'req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:41.312876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] The token cache is being invalidate because project 51e17aac08644c188e3fdc4e69abe464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:41.313166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:41.313796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:41.315337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1317/2610] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:41 2022] DELETE /identity/v3/projects/51e17aac08644c188e3fdc4e69abe464 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:44.297388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:44.299110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:44.348037 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:44.378272 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f31fb-a9d1-4fce-ae48-f55b44df15c4 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:44.439709 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:44.440359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:44.440643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:44.440917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:44.442893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:44.455684 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:44.457325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:44.467823 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1294/2611] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:44.867710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:44.869174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:44.945171 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a815dbe-c73d-4848-b1d3-e395b0d3fa28 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:44.949813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:44.950529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:44.950810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:44.951121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] PATH_INFO: `/v3/users/61f6c4b875fc41a9948dc0b2b36e0e72` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:44.952211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f6c4b875fc41a9948dc0b2b36e0e72)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:44.954147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:45.003798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f6c4b875fc41a9948dc0b2b36e0e72'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.013057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f6c4b875fc41a9948dc0b2b36e0e72'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.025143 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f6c4b875fc41a9948dc0b2b36e0e72'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.026152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f6c4b875fc41a9948dc0b2b36e0e72'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.032747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f6c4b875fc41a9948dc0b2b36e0e72'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.034134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:45.039939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1318/2612] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:44 2022] DELETE /identity/v3/users/61f6c4b875fc41a9948dc0b2b36e0e72 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:45.242163 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:45.253854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d0846-9e75-4565-a86c-07af4787132c service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:45.311681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:45.312330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:45.312585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:45.312860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] PATH_INFO: `/v3/projects/0cf9cb8a74924e77b3612f9418ad0d78` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:45.313985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf9cb8a74924e77b3612f9418ad0d78)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:45.315947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:45.366680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf9cb8a74924e77b3612f9418ad0d78', 'request_id': 'req-354ca91d-13c8-4158-b544-1b9599d629ee'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.372326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf9cb8a74924e77b3612f9418ad0d78', 'request_id': 'req-354ca91d-13c8-4158-b544-1b9599d629ee'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.378387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf9cb8a74924e77b3612f9418ad0d78', 'request_id': 'req-354ca91d-13c8-4158-b544-1b9599d629ee'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.391167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] The token cache is being invalidate because project 0cf9cb8a74924e77b3612f9418ad0d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:45.391440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.392072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.393541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1295/2613] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:45 2022] DELETE /identity/v3/projects/0cf9cb8a74924e77b3612f9418ad0d78 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:45.397696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:45.399231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:45.412756 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300bef9-56e5-4fa5-ab0b-bcada5242b17 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:45.475942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:45.476851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:45.477212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:45.477584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] PATH_INFO: `/v3/users/f66ecd8ab2aa4bf2992701c532a63dc0` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:45.479498 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66ecd8ab2aa4bf2992701c532a63dc0)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:45.482236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:45.532585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66ecd8ab2aa4bf2992701c532a63dc0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.544817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66ecd8ab2aa4bf2992701c532a63dc0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.557834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66ecd8ab2aa4bf2992701c532a63dc0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.558559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66ecd8ab2aa4bf2992701c532a63dc0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.566073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-b44a1807-17e7-405b-a9d3-cbeb151e0000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66ecd8ab2aa4bf2992701c532a63dc0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.573029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1319/2614] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:45 2022] DELETE /identity/v3/users/f66ecd8ab2aa4bf2992701c532a63dc0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:45.749060 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:45.750559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:45.760333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:45.764632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ca91d-13c8-4158-b544-1b9599d629ee admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:45.827808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:45.828394 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:45.828697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:45.828954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] PATH_INFO: `/v3/projects/494a9adfea1744119f11ecb429df5590` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:45.830104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=494a9adfea1744119f11ecb429df5590)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:45.832099 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:45.886496 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494a9adfea1744119f11ecb429df5590', 'request_id': 'req-7eaf49d4-eaac-4202-a4b5-96a0840c9448'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.892653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494a9adfea1744119f11ecb429df5590', 'request_id': 'req-7eaf49d4-eaac-4202-a4b5-96a0840c9448'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.899146 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494a9adfea1744119f11ecb429df5590', 'request_id': 'req-7eaf49d4-eaac-4202-a4b5-96a0840c9448'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.913871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] The token cache is being invalidate because project 494a9adfea1744119f11ecb429df5590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:45.914155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.914788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7eaf49d4-eaac-4202-a4b5-96a0840c9448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:45.916280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1296/2615] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:45 2022] DELETE /identity/v3/projects/494a9adfea1744119f11ecb429df5590 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:45.924117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:45.924805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:45.924989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:45.955297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:45.956521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.028026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1320/2616] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:46.039284 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.039569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.039836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.060454 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:46.061694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.070387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1297/2617] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:46.074683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.084477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580f3d-c8bf-4d27-9f9d-b0ffc6ee8fe9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.089182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.089799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.090342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.090637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.091314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:33:46.091752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.093585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.105587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1321/2618] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:33:46.109967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.119656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4048a1-5303-4229-97a5-5d7efe3a33b5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.124510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.125112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.125385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.125647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.126318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.128220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.158015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1298/2619] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:46 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:46.162382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.169835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:46.173883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a50128-d054-4ab6-a553-676bb8386a86 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.236729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.237350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.237630 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.237881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.238545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.240476 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.268135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1322/2620] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:46 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:33:46.272970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.286097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9934-1df8-42a0-85c7-1501c87b8be9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.291008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.291646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.291958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.292228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.292869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.294525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.304653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1299/2621] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:33:46.308716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.318810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea25ce3-4530-4786-887f-c85268012bd3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.324109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.324711 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.325057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.325331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.325974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.327649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6848e-6bea-4b58-9fd7-1f33f36a6cb3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.338235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1323/2622] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:46.342191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.352827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d9036-bbb2-400c-964e-28774af6387f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.358160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.358854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.359138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.359411 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] PATH_INFO: `/v3/projects/562c8d16a23f4569ab3896d8f632067e/users/82d938f10c814c06b2258fcbe0a9fcbf/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.375027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=562c8d16a23f4569ab3896d8f632067e, user_id=82d938f10c814c06b2258fcbe0a9fcbf, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.377705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.393065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1300/2623] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:46 2022] PUT /identity/v3/projects/562c8d16a23f4569ab3896d8f632067e/users/82d938f10c814c06b2258fcbe0a9fcbf/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:46.403595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.403884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.404158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.432669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] MFA Rules not processed for user `82d938f10c814c06b2258fcbe0a9fcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:46.433885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.499188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1324/2624] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:46.542791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.544789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.582193 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:46.608413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d65a-6f00-4412-814b-d70479f35e45 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.612490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.613106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.613387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.613658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.622578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.624077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6c33-8773-4551-9465-dc28eb87f432 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.633192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1301/2625] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:46.649735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.718406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cd8d9-f17f-4390-be54-947d7d365124 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.723778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.724413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.724686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.724954 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] PATH_INFO: `/v3/users/82d938f10c814c06b2258fcbe0a9fcbf` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.726175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d938f10c814c06b2258fcbe0a9fcbf)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.728144 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:46.743226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:46.777414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d938f10c814c06b2258fcbe0a9fcbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:46.785658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d938f10c814c06b2258fcbe0a9fcbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:46.798068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d938f10c814c06b2258fcbe0a9fcbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:46.798823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d938f10c814c06b2258fcbe0a9fcbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:46.806270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d938f10c814c06b2258fcbe0a9fcbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:46.812473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1325/2626] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:46 2022] DELETE /identity/v3/users/82d938f10c814c06b2258fcbe0a9fcbf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:46.817259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.817571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.817843 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.838357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:46.839591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.900671 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1302/2627] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:46.912086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:46.913666 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:46.971840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bc3df-8bdf-440f-988a-63703e35ca13 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:46.977872 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:46.978745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:46.979121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:46.979492 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:46.993995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:46.996114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9081e-dae8-43c2-a3e5-3e44376d75d1 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.007013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1326/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:47.287635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.289188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:47.302175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e518a-a9ae-4347-a8d0-051b7044c524 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.307458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.308171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.308459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.308733 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] PATH_INFO: `/v3/projects/562c8d16a23f4569ab3896d8f632067e` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.309991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=562c8d16a23f4569ab3896d8f632067e)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.311940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.358682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '562c8d16a23f4569ab3896d8f632067e', 'request_id': 'req-a53af47e-ba5a-4f87-bd89-9d48b940a228'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:47.364444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '562c8d16a23f4569ab3896d8f632067e', 'request_id': 'req-a53af47e-ba5a-4f87-bd89-9d48b940a228'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:47.370664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '562c8d16a23f4569ab3896d8f632067e', 'request_id': 'req-a53af47e-ba5a-4f87-bd89-9d48b940a228'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:47.383499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] The token cache is being invalidate because project 562c8d16a23f4569ab3896d8f632067e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:47.383830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:47.384668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a53af47e-ba5a-4f87-bd89-9d48b940a228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:47.386295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1303/2629] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:47 2022] DELETE /identity/v3/projects/562c8d16a23f4569ab3896d8f632067e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:47.391484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.391772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.392106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.411853 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:47.412994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:47.474521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1327/2630] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:47.483537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.483820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.484089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.504853 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:47.506089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:47.514601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1304/2631] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:47.518804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.528489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14300-d53f-46fa-b02f-14fb44c88759 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.534268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.534922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.535198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.535485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.536158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:33:47.536456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.538124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.550358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1328/2632] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:33:47.554835 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.564566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499febc-84bf-49cb-aca3-48894f4dc764 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.570309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.570959 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.571240 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.571510 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.572184 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.574132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.603820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1305/2633] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:47.608205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.619021 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a70e7d-4b6d-4db6-8e2f-f7cabe4d95de admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.639401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:47.679268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.679806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.680089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.680358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.680999 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.682986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.705865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1329/2634] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:47.710137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.721128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7bc22-4ab8-42e3-81af-deb4ddcae66e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.726881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.727479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.727753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.728016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.728660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.730252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.740505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1306/2635] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:47.744406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.754948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d7807-a4e4-4305-ba6f-40e1f717d09f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.760973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.761576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.761855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.762127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.762802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.764411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec113d99-7a20-43d2-873a-3ab0e993f3e1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.775621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1330/2636] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:33:47.779886 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:47.790261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19283-b471-42c9-9fc4-696857d0d677 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:47.796360 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:47.797023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.797294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.797556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] PATH_INFO: `/v3/projects/9bd3a1960ce5493a805d86a7dc25c017/users/892e6237ebf54ec2b5938612403847fe/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.813255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd3a1960ce5493a805d86a7dc25c017, user_id=892e6237ebf54ec2b5938612403847fe, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:47.816006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:47.818257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:47.834901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1307/2637] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:47 2022] PUT /identity/v3/projects/9bd3a1960ce5493a805d86a7dc25c017/users/892e6237ebf54ec2b5938612403847fe/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:47.846219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281f5d-271b-4067-aeab-be288796e421 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:47.846568 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281f5d-271b-4067-aeab-be288796e421 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:47.846848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11281f5d-271b-4067-aeab-be288796e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:47.875840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-11281f5d-271b-4067-aeab-be288796e421 None None] MFA Rules not processed for user `892e6237ebf54ec2b5938612403847fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:47.877218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-11281f5d-271b-4067-aeab-be288796e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:47.941883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1331/2638] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:47.946961 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.010256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb822051-b970-429b-bdb1-9285b6c87fbc admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.015453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.016057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.016339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.016611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.017295 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.019378 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.051940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1308/2639] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:48.056056 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11281f5d-271b-4067-aeab-be288796e421 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.066840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11281f5d-271b-4067-aeab-be288796e421 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.129005 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.129631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.129910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.130182 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.130860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.132798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.155495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1332/2640] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:48 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:48.159789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.170625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891de15-770d-494e-a3f9-6956f7295103 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.176331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.176934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.177217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.177495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.178140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.179784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.190362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1309/2641] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:48.194459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.205036 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a88d2-f91a-4edc-a874-e79c62f36449 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.210160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.210793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.211082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.211356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.212013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.213584 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d47140-a755-4dc2-93b7-0fde317e620a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.224192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1333/2642] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:48.228469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.239234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e4314-da39-4221-a7e5-030351879837 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.244138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.244800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.245108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.245379 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] PATH_INFO: `/v3/projects/f6a825ba644b4efe9d4adff4210be9a3/users/c1fc5973690340468c157737d3ad5945/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.260706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a825ba644b4efe9d4adff4210be9a3, user_id=c1fc5973690340468c157737d3ad5945, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.263415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.277255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1310/2643] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:48 2022] PUT /identity/v3/projects/f6a825ba644b4efe9d4adff4210be9a3/users/c1fc5973690340468c157737d3ad5945/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:33:48.287844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.288050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.288301 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.308883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:48.318725 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] MFA Rules not processed for user `c1fc5973690340468c157737d3ad5945`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:48.320900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:48.390098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1334/2644] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:48.395030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.465392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21f48-d4e6-4c0e-abf2-a1cb625eeaf9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.470438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.471075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.471356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.471627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.472258 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.474168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.502631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1311/2645] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:48 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:48.506978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.517848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5bacd-3e40-488f-9d14-8e2675a21cd7 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.584336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.584968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.585241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.585509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.586179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.588587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.614345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1335/2646] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:48 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:48.617854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.628625 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f3cfc-b796-4193-b505-91908bcb94a6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.634036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.634658 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.634939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.635206 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.635836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.637415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.648436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1312/2647] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:48.652591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.662889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ccbd1-59c9-423d-bf00-5ed206e61af5 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.667875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.668527 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.668808 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.669080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] PATH_INFO: `/v3/projects/c341f1cf3e89417eae792ea3b94ab078/users/d2914067f880431a83bd1625258864b1/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.685046 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c341f1cf3e89417eae792ea3b94ab078, user_id=d2914067f880431a83bd1625258864b1, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.687749 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.704397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1336/2648] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:48 2022] PUT /identity/v3/projects/c341f1cf3e89417eae792ea3b94ab078/users/d2914067f880431a83bd1625258864b1/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:48.708642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.722937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431cb50-8345-440f-b850-56db0f3f0672 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.735608 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:48.781860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.782637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.782986 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.783257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.783894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.785491 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24087-1b2f-4b32-ae0f-d07842214277 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.795744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1313/2649] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:48.800304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.810858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf550a-4e9a-44ae-a0e6-93c9b7471c49 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:48.815967 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:48.816624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.816903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.817211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] PATH_INFO: `/v3/projects/c341f1cf3e89417eae792ea3b94ab078/users/d2914067f880431a83bd1625258864b1/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.819230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c341f1cf3e89417eae792ea3b94ab078, user_id=d2914067f880431a83bd1625258864b1, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:48.822242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:48.832846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1337/2650] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:48 2022] PUT /identity/v3/projects/c341f1cf3e89417eae792ea3b94ab078/users/d2914067f880431a83bd1625258864b1/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:33:48.844049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:48.844333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:48.844602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:48.873396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] MFA Rules not processed for user `d2914067f880431a83bd1625258864b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:48.874656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:48.946228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1314/2651] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:48.957623 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:48.959510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:49.019369 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef426ff-29a2-4f5c-a7ad-00aff7577c70 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:49.023479 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:49.024066 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:49.024342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:49.024615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:49.028989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:49.082546 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:49.083991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:49.093336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1338/2652] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:50.617523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:50.619057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:50.631348 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5487a3-80d7-445e-a226-298adb08842b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:50.635894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:50.636502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:50.636779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:50.637043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:50.693094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:50.694576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:50.704076 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1315/2653] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:51.254937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:51.263810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac148-d45b-4f29-b7d9-b4cb748cb931 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:51.267974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:51.268685 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:51.269011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:51.269282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:51.278541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:51.279979 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf77f4-512a-41fc-9996-08e99c7ef9b1 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:51.289149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1339/2654] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:55.235171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:55.283694 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:55.318195 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33238-2b4d-48f4-8f16-06b603fe0d5b service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:55.324053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:55.324726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:55.324971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:55.325271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] PATH_INFO: `/v3/users/892e6237ebf54ec2b5938612403847fe` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:55.326465 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] RBAC: Authorizing `identity:delete_user(user_id=892e6237ebf54ec2b5938612403847fe)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:55.328386 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:55.383337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '892e6237ebf54ec2b5938612403847fe'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:55.393880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '892e6237ebf54ec2b5938612403847fe'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:55.407654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '892e6237ebf54ec2b5938612403847fe'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:55.408251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '892e6237ebf54ec2b5938612403847fe'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:55.415996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '892e6237ebf54ec2b5938612403847fe'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:55.422398 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1316/2655] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:55 2022] DELETE /identity/v3/users/892e6237ebf54ec2b5938612403847fe => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:55.427152 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:55.427649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:55.427649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:55.448715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:55.449946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:55.514346 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1340/2656] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:55.528199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:55.529880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:55.601286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea8926e-4885-4be8-9278-a2c017e8041c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:55.605672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:55.606298 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:55.606614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:55.606889 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:55.616795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:55.618259 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:55.628517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1317/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:55.923241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:55.925111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:55.938332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2f382-3a55-4076-abb3-dbbbdb440a7d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:55.943324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:55.944054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:55.944402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:55.945096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] PATH_INFO: `/v3/projects/9bd3a1960ce5493a805d86a7dc25c017` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:55.946295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd3a1960ce5493a805d86a7dc25c017)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:55.948374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:55.998960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd3a1960ce5493a805d86a7dc25c017', 'request_id': 'req-805d4d22-8968-4a44-b6db-6be7298865ff'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.005082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd3a1960ce5493a805d86a7dc25c017', 'request_id': 'req-805d4d22-8968-4a44-b6db-6be7298865ff'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.012587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd3a1960ce5493a805d86a7dc25c017', 'request_id': 'req-805d4d22-8968-4a44-b6db-6be7298865ff'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.026511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] The token cache is being invalidate because project 9bd3a1960ce5493a805d86a7dc25c017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:56.026803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.027434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.029009 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1341/2658] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:55 2022] DELETE /identity/v3/projects/9bd3a1960ce5493a805d86a7dc25c017 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:56.033537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:56.043545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64131c58-45fa-4149-8102-f4fc750e1de5 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:56.100582 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:56.101228 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:56.101505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:56.101773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] PATH_INFO: `/v3/users/c1fc5973690340468c157737d3ad5945` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:56.102918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1fc5973690340468c157737d3ad5945)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:56.104844 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:56.106477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:56.155375 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1fc5973690340468c157737d3ad5945'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.163779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1fc5973690340468c157737d3ad5945'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.176804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1fc5973690340468c157737d3ad5945'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.177391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1fc5973690340468c157737d3ad5945'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.184319 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1fc5973690340468c157737d3ad5945'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.190269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1318/2659] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:56 2022] DELETE /identity/v3/users/c1fc5973690340468c157737d3ad5945 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:56.494963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:56.496419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:56.510532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d4d22-8968-4a44-b6db-6be7298865ff admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:56.523455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:56.572341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:56.573015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:56.573293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:56.573585 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] PATH_INFO: `/v3/projects/f6a825ba644b4efe9d4adff4210be9a3` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:56.574798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a825ba644b4efe9d4adff4210be9a3)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:56.576757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:56.624775 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a825ba644b4efe9d4adff4210be9a3', 'request_id': 'req-f731898a-ad9f-4603-b35c-de0fa67bfa29'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.630594 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a825ba644b4efe9d4adff4210be9a3', 'request_id': 'req-f731898a-ad9f-4603-b35c-de0fa67bfa29'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.637154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a825ba644b4efe9d4adff4210be9a3', 'request_id': 'req-f731898a-ad9f-4603-b35c-de0fa67bfa29'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.652035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] The token cache is being invalidate because project f6a825ba644b4efe9d4adff4210be9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:56.652295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.653082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.654669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1342/2660] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:56 2022] DELETE /identity/v3/projects/f6a825ba644b4efe9d4adff4210be9a3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:56.658921 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:56.660508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:56.674657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745c4d2-fc75-4aa5-a612-f05ab8bdc3b2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:56.733415 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:56.734052 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:56.734327 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:56.734629 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] PATH_INFO: `/v3/users/d2914067f880431a83bd1625258864b1` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:56.735784 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2914067f880431a83bd1625258864b1)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:56.737686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:56.804970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2914067f880431a83bd1625258864b1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.812029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2914067f880431a83bd1625258864b1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.824165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2914067f880431a83bd1625258864b1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.824783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2914067f880431a83bd1625258864b1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.831451 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fe81e371-d7e6-44b2-9ca5-0ce18e33ce35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2914067f880431a83bd1625258864b1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:56.832866 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:56.838908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1319/2661] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:33:56 2022] DELETE /identity/v3/users/d2914067f880431a83bd1625258864b1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:57.123869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.125420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:57.139990 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731898a-ad9f-4603-b35c-de0fa67bfa29 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.198738 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.199384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.199667 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.199939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] PATH_INFO: `/v3/projects/c341f1cf3e89417eae792ea3b94ab078` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.201064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c341f1cf3e89417eae792ea3b94ab078)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.203038 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.249127 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:57.250681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c341f1cf3e89417eae792ea3b94ab078', 'request_id': 'req-6eb80453-df00-40b6-a898-999a3448fdd2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:57.256981 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c341f1cf3e89417eae792ea3b94ab078', 'request_id': 'req-6eb80453-df00-40b6-a898-999a3448fdd2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:57.263303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c341f1cf3e89417eae792ea3b94ab078', 'request_id': 'req-6eb80453-df00-40b6-a898-999a3448fdd2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:57.277163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] The token cache is being invalidate because project c341f1cf3e89417eae792ea3b94ab078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:33:57.277450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:57.278076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-6eb80453-df00-40b6-a898-999a3448fdd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:33:57.279599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1343/2662] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:33:57 2022] DELETE /identity/v3/projects/c341f1cf3e89417eae792ea3b94ab078 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:57.284829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.285123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.285425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.306138 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:57.307429 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:57.370186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1320/2663] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:33:57.378536 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.378827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.379104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.400442 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:57.401652 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:57.410494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1344/2664] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:57.415256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.425765 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3066b-4070-4ead-b8ec-5955e791104d None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.431423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.432024 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.432349 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.432701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.433365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:33:57.433669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.435331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.448483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1321/2665] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:33:57.453113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.463192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2304-feca-48f7-b8e8-41dd864d201c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.468572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.469177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.469459 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.469729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.470398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.472380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.503560 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1345/2666] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:33:57 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:33:57.508179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.518615 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ab28bd-a64d-455f-8ce6-4a4d336c8215 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.577414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.578036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.578316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.578638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.579278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.581233 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.604211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1322/2667] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:33:57 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:33:57.608471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.619444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54b642-49fe-4059-b1c1-110c128b4261 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.625082 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.625689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.625995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.626268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.626951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.628688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.639229 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1346/2668] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:57.643687 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.654587 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1fd6a-0fd6-4abb-8420-c2d253cfb09b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.661606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.662211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.662517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.662771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.663434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.665523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cede1b-ea72-48a1-a776-9ea6d65fb3dd admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.680247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1323/2669] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:33:57.684778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.695811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5e9cf-5760-4f33-81f2-052027c304cb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.700766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.701428 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.701708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.701980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] PATH_INFO: `/v3/projects/c74ac213e1584e388ea5893ebea9032b/users/02b8459387ca46e49cbc894ec758d5e0/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.724319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74ac213e1584e388ea5893ebea9032b, user_id=02b8459387ca46e49cbc894ec758d5e0, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.727043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.742592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1347/2670] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:33:57 2022] PUT /identity/v3/projects/c74ac213e1584e388ea5893ebea9032b/users/02b8459387ca46e49cbc894ec758d5e0/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:33:57.754245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.754514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.754792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.785207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] MFA Rules not processed for user `02b8459387ca46e49cbc894ec758d5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:33:57.786417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:57.855637 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:57.860019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1324/2671] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:33:57.872292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:57.874085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:57.932746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14c736-40bf-4c4b-b37d-1019a4ca3650 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:57.936970 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:57.937610 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:57.937892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:57.938163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:57.947839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:57.949277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc3452-9e8c-4dd3-b304-6f35a905e82e service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:57.958615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1348/2672] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:07.857863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:07.954461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:34.517338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:34.584030 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da4069-b229-4d78-b4db-8243bf944cda None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:34.589462 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:34.590094 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:34.590404 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:34.590670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] PATH_INFO: `/v3/users/02b8459387ca46e49cbc894ec758d5e0` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:34.591832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b8459387ca46e49cbc894ec758d5e0)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:34.593759 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:34.644425 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b8459387ca46e49cbc894ec758d5e0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:34.654088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b8459387ca46e49cbc894ec758d5e0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:34.668110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b8459387ca46e49cbc894ec758d5e0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:34.668814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b8459387ca46e49cbc894ec758d5e0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:34.676007 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b8459387ca46e49cbc894ec758d5e0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:34.682019 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1325/2673] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:34:34 2022] DELETE /identity/v3/users/02b8459387ca46e49cbc894ec758d5e0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:34.686924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:34.687405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:34.687491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:34.707026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:34.708204 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:34.777047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1349/2674] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:34.787065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:34.788723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:34.848009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6233-9c1c-493f-9d03-3d6ecbb20ed9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:34.852091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:34.852701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:34.852976 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:34.853252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:34.863025 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:34.864485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61996-25a3-462c-9f43-1e28ea3d38a4 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:34.874933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1326/2675] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:35.159337 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.160922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:35.174445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bf5c-e505-4bf4-a622-bf3d7119d501 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.184746 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.185499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.185783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.186050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] PATH_INFO: `/v3/projects/c74ac213e1584e388ea5893ebea9032b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.187360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74ac213e1584e388ea5893ebea9032b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.189302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.237293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74ac213e1584e388ea5893ebea9032b', 'request_id': 'req-c9e449b2-9d31-4473-83fa-751aba2ff0e2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:35.243049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74ac213e1584e388ea5893ebea9032b', 'request_id': 'req-c9e449b2-9d31-4473-83fa-751aba2ff0e2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:35.249350 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74ac213e1584e388ea5893ebea9032b', 'request_id': 'req-c9e449b2-9d31-4473-83fa-751aba2ff0e2'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:35.262505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] The token cache is being invalidate because project c74ac213e1584e388ea5893ebea9032b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:34:35.262822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:35.263463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-c9e449b2-9d31-4473-83fa-751aba2ff0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:35.264974 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1350/2676] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:34:35 2022] DELETE /identity/v3/projects/c74ac213e1584e388ea5893ebea9032b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:35.270091 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.270400 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.270711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.290103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:35.291296 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:35.308074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:35.356359 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1327/2677] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:34:35.364558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.365120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.365430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.388799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:35.390007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:35.399341 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1351/2678] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:35.403931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.414013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad887-c0d6-4b57-aa6c-4140cf56aa3b None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.419751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.420367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.420644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.420919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.421578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:34:35.421887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.423544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.435079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1328/2679] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:34:35.439382 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.449250 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb436eea-6ca1-4986-8099-c06f10c83bfb None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.455231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.455868 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.456149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.456426 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.457058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.459223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.488590 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1352/2680] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:34:35 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:34:35.492810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.503134 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed08268-ed85-460e-80f2-9d559365bfd6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.567864 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.568540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.568775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.569409 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.570338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.572287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.595189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1329/2681] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:34:35 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:34:35.600139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.616219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5477f-22b4-4c2e-9280-0265739233b1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.624221 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.624804 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.625111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.625391 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.626006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.627651 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.638793 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1353/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:34:35.642702 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.653761 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1fce3-94b4-44e7-8cb6-314e6b0a086b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.659194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.659807 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.660070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.660350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.660997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.662653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829bc7d-6940-4dc5-b96c-c5e992ac5268 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.674039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1330/2683] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:34:35.678314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.688321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442195e-0de0-4485-b6c0-100c218a58d8 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.693778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.694856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.695132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.695404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] PATH_INFO: `/v3/projects/7878c5e6e1e143199681f42b5518cc1a/users/d20ba942c80848d7819aede8c0df1e7f/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.710576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7878c5e6e1e143199681f42b5518cc1a, user_id=d20ba942c80848d7819aede8c0df1e7f, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.713238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.727063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1354/2684] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:34:35 2022] PUT /identity/v3/projects/7878c5e6e1e143199681f42b5518cc1a/users/d20ba942c80848d7819aede8c0df1e7f/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:35.732260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdd4a-2775-43b6-94e6-a1b88f4c592e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.732542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdd4a-2775-43b6-94e6-a1b88f4c592e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.732832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bdd4a-2775-43b6-94e6-a1b88f4c592e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.751809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-125bdd4a-2775-43b6-94e6-a1b88f4c592e None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:35.753014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-125bdd4a-2775-43b6-94e6-a1b88f4c592e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:35.815471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1331/2685] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:35.827564 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:35.829604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:35.852655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:35.889146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112803-d387-4bba-abfb-ddc510d0f5e9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:35.893205 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:35.893815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:35.894096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:35.894393 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:35.903599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:35.905072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:35.914538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1355/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:39.905794 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:39.906417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:39.906417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:39.942949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] MFA Rules not processed for user `d20ba942c80848d7819aede8c0df1e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:39.944455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:39.962261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:40.015239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1332/2687] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:40.019876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.030521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17271f-64f5-4ef9-940f-aaf203685546 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.035915 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.036541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.036818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.037086 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.037721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.039734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.072044 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1356/2688] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:34:40 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:34:40.073205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.085731 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d46b62-7b8a-4c3e-8814-4e04e6f8462a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.146991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.147635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.147918 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.148194 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.148840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.150824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.176239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1333/2689] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:34:40 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:34:40.180302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.191962 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e49d9-6fbc-433a-be95-9c9bd2e2dd02 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.197284 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.197945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.198293 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.198668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.199421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.201191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.216975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1357/2690] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:34:40.217927 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.228693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69efb50-a157-4d27-9dfd-9600c7411ad5 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.233892 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.234618 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.234912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.235372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] PATH_INFO: `/v3/projects/4ece4b2e7bbc45dfb841474508551daa/users/89c43023f8974882bf7ed6268d62053d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.251466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ece4b2e7bbc45dfb841474508551daa, user_id=89c43023f8974882bf7ed6268d62053d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.254175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.275902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1334/2691] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:34:40 2022] PUT /identity/v3/projects/4ece4b2e7bbc45dfb841474508551daa/users/89c43023f8974882bf7ed6268d62053d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:40.280035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.290706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af1915-0d89-4511-aaf5-453bdf5882d6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.361225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.361852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.362133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.362436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.363093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.364729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2901f-78f0-4ccb-bb9f-ea2a2a92e7a8 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.375260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1358/2692] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:34:40.381147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:40.396911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8029-7b51-4110-853b-0d71a3e7a610 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:40.403160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:40.404227 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:40.404838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:40.404838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] PATH_INFO: `/v3/projects/4ece4b2e7bbc45dfb841474508551daa/users/89c43023f8974882bf7ed6268d62053d/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:40.408540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ece4b2e7bbc45dfb841474508551daa, user_id=89c43023f8974882bf7ed6268d62053d, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:40.411845 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:40.440899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1335/2693] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:34:40 2022] PUT /identity/v3/projects/4ece4b2e7bbc45dfb841474508551daa/users/89c43023f8974882bf7ed6268d62053d/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:45.234641 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ef71-4087-4b50-93ee-fda543b04c22 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:45.235174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ef71-4087-4b50-93ee-fda543b04c22 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:45.235174 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ef71-4087-4b50-93ee-fda543b04c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:45.269245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ead2ef71-4087-4b50-93ee-fda543b04c22 None None] MFA Rules not processed for user `89c43023f8974882bf7ed6268d62053d`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:45.270519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ead2ef71-4087-4b50-93ee-fda543b04c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:45.341405 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1359/2694] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:45.369372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:45.370948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:45.450070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea5c2-9c60-4183-8222-e6f18ed6a8d9 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:45.455664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:45.456282 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:45.456559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:45.456828 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:45.461995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:45.511537 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:45.513167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:45.521953 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1336/2695] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:55.338616 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:55.518065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=24 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:01.934319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:01.934858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:01.934858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:01.958851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] MFA Rules not processed for user `d20ba942c80848d7819aede8c0df1e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:01.960747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:01.969165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1360/2696] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:02.011408 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:02.013080 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:02.071254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2e986-184c-4564-ab18-f603b0c01fb6 service nova] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:02.075190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:02.075793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:02.076071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:02.076341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:02.078559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:02.093459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:02.094981 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:02.106470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1337/2697] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:02.414757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:02.416323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:02.474120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4b7e6-5621-4f77-bfd9-d00393a5a286 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:02.478094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:02.478723 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:02.478998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:02.479259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:02.481113 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:02.491002 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:02.492429 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:02.500396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1361/2698] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:03.016236 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:03.026036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8442ea-fce0-4dd5-aba6-68c6a3cbcec1 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:03.030243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:03.031299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:03.031746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:03.032135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:03.035087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:03.052209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:03.053652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:03.065015 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1338/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:35:03.998141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:03.999774 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:04.068324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe7f9-4066-4d58-88ad-e50ff163908f service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:04.074235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:04.074880 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:04.075161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:04.075431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:04.077347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:04.091958 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:04.093604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:04.104908 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1362/2700] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:08.897116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:08.906549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bfc96-ebcf-4600-8c28-ac9759d865dd service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:08.910381 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:08.911087 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:08.911452 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:08.911789 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:08.913674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:08.924205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:08.925768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:08.934632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1339/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:14.100114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=61 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:18.930512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=38 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:20.762188 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:20.763812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:20.781847 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b1a5d-96f6-4dd5-bcfd-de27df8eaf06 service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:20.787526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:20.788147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:20.788430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:20.788701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:20.790519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:20.802944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:20.804548 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:20.815120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1363/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:35:23.154437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:23.163314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669fd8a-7617-4c08-a56c-b9e511caec40 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:23.167269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:23.167871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:23.168144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:23.168412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:23.177654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:23.179109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:23.188619 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1340/2703] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:26.452894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:26.467101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892b893-ffef-4204-8358-194beaeaac95 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:26.472198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:26.472802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:26.473081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:26.473345 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:26.475538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:26.542195 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:26.543675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:26.553559 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1364/2704] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:29.705248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:29.706781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:29.721278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be932e-2983-4d62-8d1e-64704ef766c7 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:29.726597 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:29.727224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:29.727503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:29.727773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] PATH_INFO: `/v3/users/87c4f9c4b9334b66ac89e01a278436bc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:29.728930 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c4f9c4b9334b66ac89e01a278436bc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:29.731049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:29.780273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c4f9c4b9334b66ac89e01a278436bc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:29.789169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c4f9c4b9334b66ac89e01a278436bc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:29.801142 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c4f9c4b9334b66ac89e01a278436bc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:29.801713 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c4f9c4b9334b66ac89e01a278436bc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:29.809448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9431bdce-9be5-4d1b-9d90-86006edff477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c4f9c4b9334b66ac89e01a278436bc'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:29.815885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1341/2705] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:35:29 2022] DELETE /identity/v3/users/87c4f9c4b9334b66ac89e01a278436bc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:35:29.997153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.008522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521aef-4535-4370-88ae-29d542dc73e1 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.065918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.066607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.066890 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.067149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] PATH_INFO: `/v3/projects/b3fa762826924accaa8012d7ceaf17eb` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.068734 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fa762826924accaa8012d7ceaf17eb)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.070706 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.121480 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fa762826924accaa8012d7ceaf17eb', 'request_id': 'req-ecf89300-660b-476f-97d7-183193c01b80'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:30.127412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fa762826924accaa8012d7ceaf17eb', 'request_id': 'req-ecf89300-660b-476f-97d7-183193c01b80'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:30.133956 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fa762826924accaa8012d7ceaf17eb', 'request_id': 'req-ecf89300-660b-476f-97d7-183193c01b80'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:30.148316 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] The token cache is being invalidate because project b3fa762826924accaa8012d7ceaf17eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:35:30.148658 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:30.149246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ecf89300-660b-476f-97d7-183193c01b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:35:30.150769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1365/2706] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:35:29 2022] DELETE /identity/v3/projects/b3fa762826924accaa8012d7ceaf17eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:35:30.157334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.157617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.157894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.179455 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:30.180690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:30.251693 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1342/2707] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:30.256139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.256664 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.256947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.286552 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:30.287780 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:30.295707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1366/2708] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:30.299994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.309698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2faf-0759-4cb3-87ba-09870f3970c1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.314399 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.315004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.315283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.315556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.316213 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:35:30.316521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.318547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.330082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1343/2709] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:35:30.334296 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.339932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:30.343983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28be64a-70a4-45f8-a92e-55dca1e199cc None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.350291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.350934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.351225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.351503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.352161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.355003 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.385712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1367/2710] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:35:30 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:35:30.389947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.400316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d99cdf-06ce-4b32-b269-27f9e63dcc35 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.446448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:30.461557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.462088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.462391 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.462689 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.463322 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.465245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.488774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1344/2711] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:35:30 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:35:30.493057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.503647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8ff65-8749-4c22-9389-bdb14f935217 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.508799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.509402 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.509682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.509951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.510710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.512321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.522329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1368/2712] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:35:30.526135 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.536879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00910d4a-f787-45ed-8b4a-874a7e480f10 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.541965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.542716 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.543064 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.543389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.544107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.545833 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eebcae-2ae5-4fbd-8229-7a92ae26c0a1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.556638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1345/2713] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:35:30.560941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.571562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedeae8-2468-4385-a943-941c561798bd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.576365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.577026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.577311 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.577582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] PATH_INFO: `/v3/projects/be0640c4e0584998a646e45565347687/users/6c42757a041b4831abd7a7ba66cbe933/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.592213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0640c4e0584998a646e45565347687, user_id=6c42757a041b4831abd7a7ba66cbe933, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.594945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.608815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1369/2714] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:35:30 2022] PUT /identity/v3/projects/be0640c4e0584998a646e45565347687/users/6c42757a041b4831abd7a7ba66cbe933/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:35:30.616715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1f21-42bf-46ce-a522-18938ca10dfc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.617111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1f21-42bf-46ce-a522-18938ca10dfc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.617492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1f21-42bf-46ce-a522-18938ca10dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.643888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-645c1f21-42bf-46ce-a522-18938ca10dfc None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:30.645057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-645c1f21-42bf-46ce-a522-18938ca10dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:30.706011 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1346/2715] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:30.716012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:30.718030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:30.780450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11821563-66b6-4405-9f74-fc9a7df86aa1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:30.784551 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:30.785208 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:30.785489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:30.785757 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:30.795943 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:30.797524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:30.807424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1370/2716] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:34.741196 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15ca8-4b4f-4c66-b001-4c06eb92368d None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:34.741715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15ca8-4b4f-4c66-b001-4c06eb92368d None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:34.741715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15ca8-4b4f-4c66-b001-4c06eb92368d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:34.770603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-fed15ca8-4b4f-4c66-b001-4c06eb92368d None None] MFA Rules not processed for user `6c42757a041b4831abd7a7ba66cbe933`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:34.771751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fed15ca8-4b4f-4c66-b001-4c06eb92368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:34.841323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1347/2717] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:34.855787 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:34.857331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:34.951825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b693fb-0a06-4c9a-9484-21ee403e317b service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:34.957401 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:34.958037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:34.958318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:34.958622 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:34.967466 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:34.969163 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:34.979372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1371/2718] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:36.297797 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590779d0-194a-415f-896f-70f9d25237dc None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:36.298270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590779d0-194a-415f-896f-70f9d25237dc None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:36.298356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590779d0-194a-415f-896f-70f9d25237dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:36.318601 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-590779d0-194a-415f-896f-70f9d25237dc None None] MFA Rules not processed for user `6c42757a041b4831abd7a7ba66cbe933`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:35:36.319832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-590779d0-194a-415f-896f-70f9d25237dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:36.328350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1348/2719] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:35:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:35:44.974991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=82,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:46.312197 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:48.697164 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:48.698960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:48.714073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b3cbd-a9c1-4112-81e3-feee36ea3709 service nova] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:48.719837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:48.720473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:48.721026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:48.721026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:48.722862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:48.735079 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:48.736742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a00716-7600-4806-a4a3-5d704988c6c8 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:48.747136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1372/2720] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:58.743020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:27.563965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590779d0-194a-415f-896f-70f9d25237dc None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:27.565865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-590779d0-194a-415f-896f-70f9d25237dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:27.626239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590779d0-194a-415f-896f-70f9d25237dc None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:27.630388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:27.631013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:27.631273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:27.631571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:27.633492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:27.693745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:27.695224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43d732-383c-4f9c-9ad2-0ba54a02471a service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:27.704779 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1349/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:36:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:36:34.041326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b194b-1f50-46b1-8005-8165a887392b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:34.041859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b194b-1f50-46b1-8005-8165a887392b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:34.041859 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b194b-1f50-46b1-8005-8165a887392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:34.062672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-705b194b-1f50-46b1-8005-8165a887392b None None] MFA Rules not processed for user `6c42757a041b4831abd7a7ba66cbe933`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:36:34.063906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-705b194b-1f50-46b1-8005-8165a887392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:34.071632 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1373/2722] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:36:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:36:37.700203 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=48 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:44.056357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:28.383878 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:28.384413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:28.384413 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:28.405188 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] MFA Rules not processed for user `d20ba942c80848d7819aede8c0df1e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:28.406459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:28.461521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1350/2723] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:28.637191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b194b-1f50-46b1-8005-8165a887392b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:28.638673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-705b194b-1f50-46b1-8005-8165a887392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:28.651175 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b194b-1f50-46b1-8005-8165a887392b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:28.654895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:28.655526 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:28.655806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:28.656076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:28.657985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:28.671275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:28.672742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:28.683591 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1374/2724] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:31.977470 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:31.979216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:32.093097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416894b-5d2c-4b0a-8496-73bff3d6b0a0 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.111885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:32.113670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:32.121782 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.122460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.122741 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.123945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.129219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c853-a90d-4bc0-aee7-dcc97d8ef7a1 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.131168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:32.135807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.136502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.136789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.137062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] PATH_INFO: `/v3/users/b07b8b251aaa456c9916c41f9a0b77e3` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.138183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07b8b251aaa456c9916c41f9a0b77e3)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.140257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:32.146016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.148277 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:32.166872 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1351/2725] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:32.216214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07b8b251aaa456c9916c41f9a0b77e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.227273 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07b8b251aaa456c9916c41f9a0b77e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.250089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07b8b251aaa456c9916c41f9a0b77e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.251024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07b8b251aaa456c9916c41f9a0b77e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.261926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-3dbe32c0-ea25-4f16-a46b-e20c4eb958f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07b8b251aaa456c9916c41f9a0b77e3'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.269368 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1375/2726] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:32 2022] DELETE /identity/v3/users/b07b8b251aaa456c9916c41f9a0b77e3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:32.461704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:32.480603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4bb56-c076-45ca-bf2d-7a7e53af67c8 service placement] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.564901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.566198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.566517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.566813 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] PATH_INFO: `/v3/projects/fdc8b1b561f247ed84babfad941e55f6` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.571145 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc8b1b561f247ed84babfad941e55f6)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.573672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:32.627487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:32.643061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc8b1b561f247ed84babfad941e55f6', 'request_id': 'req-98a39e24-7acd-4b39-b427-d4b799131027'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.650320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc8b1b561f247ed84babfad941e55f6', 'request_id': 'req-98a39e24-7acd-4b39-b427-d4b799131027'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.658783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc8b1b561f247ed84babfad941e55f6', 'request_id': 'req-98a39e24-7acd-4b39-b427-d4b799131027'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.676189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] The token cache is being invalidate because project fdc8b1b561f247ed84babfad941e55f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:32.676762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.677514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98a39e24-7acd-4b39-b427-d4b799131027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:32.683700 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1352/2727] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:32 2022] DELETE /identity/v3/projects/fdc8b1b561f247ed84babfad941e55f6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:32.685231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.685516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.685812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.706389 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:32.707607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:32.769126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1376/2728] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:32.777056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.777829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.777829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.800020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:32.801251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:32.809781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1353/2729] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:32.814074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:32.824243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f7211-b2f6-46f9-8c14-a60d354b679d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.829120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.829807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.830124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.830451 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.831190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:37:32.831485 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.833151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:32.845646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1377/2730] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:37:32.850516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:32.860031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3645561-64c7-440d-aca3-c0036232ded6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.864834 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.865444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.865723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.865991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.866681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.868603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:32.898280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1354/2731] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:32 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:32.902828 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:32.913075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77583ab-26d9-4f17-8da0-545f5d899e3a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:32.957925 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:32.974419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:32.975054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:32.975363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:32.975631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:32.976430 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:32.978848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.002352 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1378/2732] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:32 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:33.006603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.017014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e425-a003-42f1-8205-790441fb9133 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.022467 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.023266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.023602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.023954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.024664 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.026456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.036871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1355/2733] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:33.041181 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.051631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffd701-d116-4bf1-9345-e20296f88cce admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.056336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.056951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.057267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.057519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.058186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.059845 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f02481-22ab-4aef-ba4f-3c0be2c57246 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.074412 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1379/2734] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:37:33.075744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.087102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5164f-9dab-4d0d-b37e-5ed44b3c2f60 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.092050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.092803 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.093111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.093423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] PATH_INFO: `/v3/projects/c4d470775e084aeeb89208192065c751/users/eaeacea7bb3a450f8356edb094e2efc0/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.112669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d470775e084aeeb89208192065c751, user_id=eaeacea7bb3a450f8356edb094e2efc0, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.115941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.134647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1356/2735] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:33 2022] PUT /identity/v3/projects/c4d470775e084aeeb89208192065c751/users/eaeacea7bb3a450f8356edb094e2efc0/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:33.144784 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.146356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.146680 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.177977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] MFA Rules not processed for user `eaeacea7bb3a450f8356edb094e2efc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:33.179505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:33.270006 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.270218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1380/2736] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:37:33.356536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c150d-45e3-4953-8473-3b7b3d3d5bb2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.361508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.362136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.362443 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.362775 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.363368 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.365339 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.397943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1357/2737] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:33 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:37:33.402126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.415109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfd178-c368-44ca-9b64-6e8855b7a662 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.477803 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.478463 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.478743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.479014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.479693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.481626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.507540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1381/2738] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:33 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:37:33.511711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.524021 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42ac31-4052-4cdf-8e6d-d7b5a9430eed admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.531069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.531603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.531893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.532168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.532830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.534484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.547020 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1358/2739] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:37:33.551118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.561924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91bca1-f74e-4f74-8b42-8f8637a01682 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.575169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.576060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.576410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.576751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] PATH_INFO: `/v3/projects/e7891d7940fb436db7940673120f9681/users/97bc791ad5e245d491b09030a0f66453/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.592911 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7891d7940fb436db7940673120f9681, user_id=97bc791ad5e245d491b09030a0f66453, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.595856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.610154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1382/2740] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:33 2022] PUT /identity/v3/projects/e7891d7940fb436db7940673120f9681/users/97bc791ad5e245d491b09030a0f66453/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:33.613994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.624073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0b9c7-cc70-4d32-a1a0-bca991828b6a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.675648 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:33.710810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.711511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.712551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.712551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.712750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.714686 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c1ac5-ce51-4e4f-83c6-e5ecb2bbb85e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.726234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1359/2741] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:33.730001 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.741601 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbc312-8020-4485-bd6a-ad2aae030773 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:33.746443 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:33.747136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.747416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.747716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] PATH_INFO: `/v3/projects/e7891d7940fb436db7940673120f9681/users/97bc791ad5e245d491b09030a0f66453/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.750081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7891d7940fb436db7940673120f9681, user_id=97bc791ad5e245d491b09030a0f66453, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:33.752701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:33.769425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1383/2742] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:33 2022] PUT /identity/v3/projects/e7891d7940fb436db7940673120f9681/users/97bc791ad5e245d491b09030a0f66453/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:37:33.780846 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:33.781130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:33.781431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:33.842681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] MFA Rules not processed for user `97bc791ad5e245d491b09030a0f66453`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:33.844242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:33.963495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1360/2743] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:33.963957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:33.966057 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:33.979737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:34.065856 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b6fd6-e84a-4e0b-9faa-5352eb43963f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:34.069686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:34.070318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:34.070694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:34.073671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:34.084325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:34.086084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733983d-2cf6-4665-8660-3bf948ce02f8 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:34.100640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1384/2744] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:34.191473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:34.202631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01935ea7-cf2a-4a07-bd21-9fb1e641bc60 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:34.206757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:34.207622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:34.208103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:34.208568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:34.285996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:34.287521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:34.297119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1361/2745] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:36.904178 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:36.904655 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:36.904742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:36.923913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] MFA Rules not processed for user `6c42757a041b4831abd7a7ba66cbe933`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:36.925110 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:36.980929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1385/2746] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:37.335501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:37.337018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:37.388531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=100 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:37.394838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9a3f5-7c2e-461e-8e9e-4ae8ec6b0846 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:37.398652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:37.399278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:37.399557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:37.399827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:37.401647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:37.411448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:37.412873 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:37.421642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1362/2747] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:40.584615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:40.586215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:40.679993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9457b-8c20-44cc-bb65-8e63982015cd None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:40.686222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:40.686870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:40.687148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:40.687414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:40.689474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:40.701545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:40.703107 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7049c-5815-4ece-883f-0e4dc72fc1ce service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:40.712992 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1386/2748] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:44.904199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:44.987594 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd907d3-8b6d-40cb-832f-3a13e01beeb9 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:44.992472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:44.993838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:44.993838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:44.993838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] PATH_INFO: `/v3/users/eaeacea7bb3a450f8356edb094e2efc0` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:44.994907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaeacea7bb3a450f8356edb094e2efc0)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:44.996913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:45.052654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaeacea7bb3a450f8356edb094e2efc0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.061048 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaeacea7bb3a450f8356edb094e2efc0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.073885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaeacea7bb3a450f8356edb094e2efc0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.074563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaeacea7bb3a450f8356edb094e2efc0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.081275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaeacea7bb3a450f8356edb094e2efc0'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.087123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1363/2749] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:44 2022] DELETE /identity/v3/users/eaeacea7bb3a450f8356edb094e2efc0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:37:45.091878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:45.092329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:45.092486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:45.112310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:45.113544 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:45.123410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:45.172384 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1387/2750] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:45.182499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:45.184033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:45.241054 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2bad-9212-4ade-9a9f-95521860f8ca admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:45.244722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:45.245279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:45.245554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:45.245821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:45.254768 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:45.256200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:45.265821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1364/2751] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:45.593357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:45.595809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:45.620271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8f0f3-f396-48df-8448-1ff21e14c366 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:45.627167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:45.627989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:45.628314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:45.628617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] PATH_INFO: `/v3/projects/c4d470775e084aeeb89208192065c751` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:45.630030 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d470775e084aeeb89208192065c751)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:45.632140 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:45.700014 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d470775e084aeeb89208192065c751', 'request_id': 'req-ab00678d-7141-4241-b450-5cc820f290fd'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.718309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d470775e084aeeb89208192065c751', 'request_id': 'req-ab00678d-7141-4241-b450-5cc820f290fd'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.726186 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d470775e084aeeb89208192065c751', 'request_id': 'req-ab00678d-7141-4241-b450-5cc820f290fd'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.742032 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] The token cache is being invalidate because project c4d470775e084aeeb89208192065c751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:45.742499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.743607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.745764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1388/2752] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:45 2022] DELETE /identity/v3/projects/c4d470775e084aeeb89208192065c751 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:45.750278 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:45.767728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a8a44-89c7-4e40-b4ee-46a57221b62f service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:45.778466 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:45.840109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:45.840917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:45.841257 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:45.841539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] PATH_INFO: `/v3/users/97bc791ad5e245d491b09030a0f66453` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:45.842715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bc791ad5e245d491b09030a0f66453)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:45.844691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:45.900622 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bc791ad5e245d491b09030a0f66453'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.909234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bc791ad5e245d491b09030a0f66453'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.923919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bc791ad5e245d491b09030a0f66453'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.924610 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bc791ad5e245d491b09030a0f66453'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.932940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-27685d0c-b941-487b-a7b6-878f3b84d10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bc791ad5e245d491b09030a0f66453'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:45.939565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1365/2753] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:45 2022] DELETE /identity/v3/users/97bc791ad5e245d491b09030a0f66453 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:46.220741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.222385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:46.240075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00678d-7141-4241-b450-5cc820f290fd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.302185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.302760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.303042 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.303314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] PATH_INFO: `/v3/projects/e7891d7940fb436db7940673120f9681` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.304446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7891d7940fb436db7940673120f9681)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.306510 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.347795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:46.355069 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7891d7940fb436db7940673120f9681', 'request_id': 'req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:46.361004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7891d7940fb436db7940673120f9681', 'request_id': 'req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:46.367177 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7891d7940fb436db7940673120f9681', 'request_id': 'req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:46.381026 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] The token cache is being invalidate because project e7891d7940fb436db7940673120f9681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:46.381318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:46.381953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-dfb9a58d-ca3b-4a60-acbe-fec20f9f05c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:46.383517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1389/2754] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:46 2022] DELETE /identity/v3/projects/e7891d7940fb436db7940673120f9681 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:46.389265 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.389546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.389818 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.409696 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:46.411077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:46.472092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1366/2755] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:46.479871 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.480159 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.480432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.500151 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:46.501364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:46.510556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1390/2756] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:46.514469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.524352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5ce7-d0c1-4dac-a5f0-5c93ea618977 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.529009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.529652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.529955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.530205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.530900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:37:46.531207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.532806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.547928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1367/2757] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:37:46.552295 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.561531 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ec80f-8cfb-4a7e-a557-7e71794ddb38 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.566714 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.567327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.567604 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.567875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.568511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.570483 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.601922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1391/2758] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:46.606774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.619649 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b52f7e-b234-41e4-b1fd-c1614de3df85 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.632688 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:46.678934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.679572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.679850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.680154 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.680785 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.682740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.706102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1368/2759] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:37:46.710646 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.720900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cac34-a482-4d79-8cb0-203dbe05e5cf admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.725668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.726282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.726617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.726889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.727615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.729243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.739215 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1392/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:46.743187 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.753216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c7ad-3ec6-40c1-9096-4c56af0882ae admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.758245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.758960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.759241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.759512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.760162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.761758 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d313a6c-541c-4715-9d0d-064cb41da514 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.778457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1369/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:46.782983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:46.793532 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94fd14-4459-4d12-be83-8311f19baa42 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:46.798540 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:46.799187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.799468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.799737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] PATH_INFO: `/v3/projects/2a5568e230d4432a93efde142513f231/users/119c58715cfb4123bad2df6df7165d0d/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.815482 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5568e230d4432a93efde142513f231, user_id=119c58715cfb4123bad2df6df7165d0d, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:46.818068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:46.833161 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1393/2762] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:46 2022] PUT /identity/v3/projects/2a5568e230d4432a93efde142513f231/users/119c58715cfb4123bad2df6df7165d0d/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:46.843426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:46.843710 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:46.843988 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:46.876642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] MFA Rules not processed for user `119c58715cfb4123bad2df6df7165d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:46.877860 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:46.962933 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1370/2763] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:46.967509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.029625 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa80f6-9c04-4204-8b2e-791fc53c40e9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.034303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.034946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.035230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.035501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.036142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.038073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.068449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1394/2764] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:46 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:47.073422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.089737 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43042-7531-4a6b-974a-bb75494ab515 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.146902 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.147554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.147836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.148112 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.148762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.150820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.174287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1371/2765] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:47 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:37:47.178693 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.189134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7dd2-630c-4612-ad9f-d78cc288fe65 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.195189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.195789 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.196064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.196358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.196975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.198619 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.213020 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1395/2766] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:47.214318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.224900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc2ceb-0a99-4583-b0e6-521ea5fc4f6f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.235103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.235804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.236082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.236355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] PATH_INFO: `/v3/projects/0e7d456e051847038b4157f4c63e0b54/users/e2d495c192914998a09f2625c5af828c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.250571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:47.254013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7d456e051847038b4157f4c63e0b54, user_id=e2d495c192914998a09f2625c5af828c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.257500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.272069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1372/2767] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:47 2022] PUT /identity/v3/projects/0e7d456e051847038b4157f4c63e0b54/users/e2d495c192914998a09f2625c5af828c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:47.276376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.286874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba3a20-0fee-473a-8b9e-2c3627700b35 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.328431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:47.349509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.350272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.350770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.351047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.351698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.353403 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3b8e8-df53-4e20-931b-c34e3c5b4ca5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.364421 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1396/2768] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:47.369092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.379611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56a899-cc35-478f-a2f7-d993a08a2037 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.384512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.385176 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.385457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.385730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] PATH_INFO: `/v3/projects/0e7d456e051847038b4157f4c63e0b54/users/e2d495c192914998a09f2625c5af828c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.387705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7d456e051847038b4157f4c63e0b54, user_id=e2d495c192914998a09f2625c5af828c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.390385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.401545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1373/2769] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:47 2022] PUT /identity/v3/projects/0e7d456e051847038b4157f4c63e0b54/users/e2d495c192914998a09f2625c5af828c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:47.413283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.413566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.413839 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.443697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] MFA Rules not processed for user `e2d495c192914998a09f2625c5af828c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:47.444913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:47.513109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1397/2770] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:47.525132 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.526912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:47.586854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de70ff3-acf6-4980-81e3-9bf152bd3ffa admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.590488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.591106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.591382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.591657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.600626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.602056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.611852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1374/2771] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:47.669876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:47.671424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:47.685836 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff301991-5314-45a9-befc-68b4954aba4b None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:47.689841 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:47.691061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:47.691435 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:47.691769 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:47.757131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:47.758827 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2687c9-f443-44cc-bfe2-c1b19db4464a service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:47.768565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1398/2772] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:51.286056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:51.351849 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe483-c007-413b-8fe6-fbdefd80428f service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:51.356563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:51.357175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:51.357480 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:51.357857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] PATH_INFO: `/v3/users/119c58715cfb4123bad2df6df7165d0d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:51.359003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=119c58715cfb4123bad2df6df7165d0d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:51.361513 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:51.412720 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119c58715cfb4123bad2df6df7165d0d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.422198 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119c58715cfb4123bad2df6df7165d0d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.434994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119c58715cfb4123bad2df6df7165d0d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.435674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119c58715cfb4123bad2df6df7165d0d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.443305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119c58715cfb4123bad2df6df7165d0d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.445987 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:51.451372 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1375/2773] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:51 2022] DELETE /identity/v3/users/119c58715cfb4123bad2df6df7165d0d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:51.455977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:51.456497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:51.456497 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:51.477504 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:51.478754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:51.531372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=100 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:51.541388 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1399/2774] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:51.550326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:51.551876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:51.610231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27b55d-ee11-49bc-962b-d651117f2936 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:51.613970 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:51.614631 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:51.614891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:51.615161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:51.624540 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:51.626070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:51.635729 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1376/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:51.910823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:51.912338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:51.927185 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c4840-31da-43e1-bbff-5c816f488901 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:51.933691 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:51.934470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:51.934817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:51.935922 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] PATH_INFO: `/v3/projects/2a5568e230d4432a93efde142513f231` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:51.937486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5568e230d4432a93efde142513f231)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:51.940191 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:51.992589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5568e230d4432a93efde142513f231', 'request_id': 'req-0d82833c-677d-4746-8503-8a71e090fe7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:51.999603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5568e230d4432a93efde142513f231', 'request_id': 'req-0d82833c-677d-4746-8503-8a71e090fe7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.006955 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5568e230d4432a93efde142513f231', 'request_id': 'req-0d82833c-677d-4746-8503-8a71e090fe7e'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.022087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] The token cache is being invalidate because project 2a5568e230d4432a93efde142513f231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:52.022411 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.023093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.024849 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1400/2776] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:51 2022] DELETE /identity/v3/projects/2a5568e230d4432a93efde142513f231 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:52.029119 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:52.038621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8dc83-b124-4de4-b525-a9bafe6cabd3 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:52.100283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:52.100936 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.101218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.101498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] PATH_INFO: `/v3/users/e2d495c192914998a09f2625c5af828c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.102717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d495c192914998a09f2625c5af828c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:52.105268 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:52.158316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d495c192914998a09f2625c5af828c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.169488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d495c192914998a09f2625c5af828c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.184410 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d495c192914998a09f2625c5af828c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.187035 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d495c192914998a09f2625c5af828c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.193900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-3e09e01c-8ce7-4ad9-b133-8ba9729a3818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d495c192914998a09f2625c5af828c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.200362 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1377/2777] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:52 2022] DELETE /identity/v3/users/e2d495c192914998a09f2625c5af828c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:37:52.500993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:52.503033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:52.518971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82833c-677d-4746-8503-8a71e090fe7e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:52.586672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:52.587324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.587600 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.587866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] PATH_INFO: `/v3/projects/0e7d456e051847038b4157f4c63e0b54` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.588976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7d456e051847038b4157f4c63e0b54)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:52.590977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:52.644114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7d456e051847038b4157f4c63e0b54', 'request_id': 'req-02b2dc88-604e-48b0-884a-d1a6d6d09b04'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.649831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7d456e051847038b4157f4c63e0b54', 'request_id': 'req-02b2dc88-604e-48b0-884a-d1a6d6d09b04'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.656806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7d456e051847038b4157f4c63e0b54', 'request_id': 'req-02b2dc88-604e-48b0-884a-d1a6d6d09b04'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.671187 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] The token cache is being invalidate because project 0e7d456e051847038b4157f4c63e0b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:52.671478 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.672126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-02b2dc88-604e-48b0-884a-d1a6d6d09b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:52.673736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1401/2778] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:52 2022] DELETE /identity/v3/projects/0e7d456e051847038b4157f4c63e0b54 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:52.680175 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.680699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.680815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.701606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:52.702914 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:52.720654 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:52.768095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1378/2779] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:52.776136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.776419 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.776688 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.796903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:52.798093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:52.806801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1402/2780] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:52.810667 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:52.821189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3ef5-801d-41c4-adf3-a8080df088c1 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:52.825913 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:52.826668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.826967 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.827234 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.828086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:37:52.828199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:52.829776 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:52.842098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1379/2781] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:37:52.846983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:52.856889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733336da-8cdd-4962-a26b-23ca3090ad19 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:52.862505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:52.863235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.863607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.863916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.864637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:52.867663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:52.873535 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:52.901062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1403/2782] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:52 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:52.905297 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:52.915945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c08537-a6b4-48f4-9aa5-4cb09b82c6b2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:52.971057 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:52.971661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:52.971939 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:52.972208 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:52.972836 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:52.974802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:52.998657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1380/2783] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:52 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:37:53.003048 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:53.014467 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23c99-02bc-4c96-acf3-068db966e5a9 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:53.019461 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:53.020317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:53.020605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:53.020881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:53.021525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:53.023180 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:53.034136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1404/2784] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:53.039938 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:53.055151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e682aa5-4616-4e4b-a348-80c8a4bb9f8a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:53.061127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:53.061732 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:53.062009 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:53.062279 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:53.062946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:53.064557 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c049c-dd33-47a5-b3d8-f6363c2d16b3 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:53.076067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1381/2785] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:53.080294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:53.091278 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc2347-34de-44fe-8b7c-46e6163a0d45 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:53.096348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:53.096995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:53.097267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:53.097537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] PATH_INFO: `/v3/projects/0e12c546b12e4e6fb765c9cc61f4e8c9/users/4a65922da34b4b6b8b120ad0c0fd3ea4/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:53.112120 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e12c546b12e4e6fb765c9cc61f4e8c9, user_id=4a65922da34b4b6b8b120ad0c0fd3ea4, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:53.114829 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:53.130063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1405/2786] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:53 2022] PUT /identity/v3/projects/0e12c546b12e4e6fb765c9cc61f4e8c9/users/4a65922da34b4b6b8b120ad0c0fd3ea4/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:53.142300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:53.142690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:53.143033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:53.171217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] MFA Rules not processed for user `4a65922da34b4b6b8b120ad0c0fd3ea4`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:53.172446 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:53.242626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1382/2787] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:37:53.254243 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:53.256184 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:53.317012 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91424a14-00f9-49f3-9e22-9b1d4034a2ee admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:53.320991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:53.321611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:53.321891 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:53.322160 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:53.331092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:53.332470 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be90ae-3c7b-437b-8b17-b870b0c7038e service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:53.341592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1406/2788] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:55.940488 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:55.961589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:56.008291 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe780d1-e3d9-40c9-896d-3892ccf5d499 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:56.014481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:56.015177 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.015457 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.015727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] PATH_INFO: `/v3/users/4a65922da34b4b6b8b120ad0c0fd3ea4` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.017072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a65922da34b4b6b8b120ad0c0fd3ea4)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:56.019071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:56.071056 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a65922da34b4b6b8b120ad0c0fd3ea4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.079971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a65922da34b4b6b8b120ad0c0fd3ea4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.092552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a65922da34b4b6b8b120ad0c0fd3ea4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.093172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a65922da34b4b6b8b120ad0c0fd3ea4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.100260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a65922da34b4b6b8b120ad0c0fd3ea4'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.106293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1383/2789] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:55 2022] DELETE /identity/v3/users/4a65922da34b4b6b8b120ad0c0fd3ea4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:56.111220 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.111704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.111791 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.132117 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:56.133320 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:56.198846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1407/2790] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:56.209333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:56.211039 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:56.270183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128bfa4-cf5a-4853-afa2-4c0b315a207d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:56.275355 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:56.276287 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.276660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.277018 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.287275 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:56.288725 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a56396-928a-4341-b196-36d39890d839 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:56.298565 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1384/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:56.586213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:56.588064 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:56.603703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4ff31-a5bc-48be-8dbf-8c0b70c71be2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:56.610747 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:56.611374 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.611721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.611932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] PATH_INFO: `/v3/projects/0e12c546b12e4e6fb765c9cc61f4e8c9` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.613045 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e12c546b12e4e6fb765c9cc61f4e8c9)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:56.615022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:56.664565 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:56.666216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e12c546b12e4e6fb765c9cc61f4e8c9', 'request_id': 'req-615199ea-a598-4e7d-acac-169aabcc2593'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.673722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e12c546b12e4e6fb765c9cc61f4e8c9', 'request_id': 'req-615199ea-a598-4e7d-acac-169aabcc2593'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.680067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e12c546b12e4e6fb765c9cc61f4e8c9', 'request_id': 'req-615199ea-a598-4e7d-acac-169aabcc2593'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.694272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] The token cache is being invalidate because project 0e12c546b12e4e6fb765c9cc61f4e8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:56.694637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.695268 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-615199ea-a598-4e7d-acac-169aabcc2593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:56.696778 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1408/2792] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:56 2022] DELETE /identity/v3/projects/0e12c546b12e4e6fb765c9cc61f4e8c9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:56.702110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.702354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.702657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.723369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:56.724581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:56.762536 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:56.795932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1385/2793] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:56.800944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.801226 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.801494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.822237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:56.823502 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:56.832477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1409/2794] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:56.837316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:56.852305 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7f850-9273-471c-9e45-183b55bed5a6 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:56.859385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:56.859997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.860283 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.860550 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.861201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:37:56.861505 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:56.863787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:56.876084 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1386/2795] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:37:56.880572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:56.890258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3bbfb-dce6-4ce6-8721-7453125fc748 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:56.896259 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:56.896882 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:56.897166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:56.897425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:56.898063 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:56.900047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:56.929948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1410/2796] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:56 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:56.934354 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:56.945683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d905c3c-7c7a-4a01-8016-5990d8dc6115 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.011534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.012211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.012495 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.012767 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.013485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.015560 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.043755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1387/2797] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:56 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:57.047843 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.058547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7c3ca-b29c-46ec-8486-fb212ef6196b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.064596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.065550 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.065823 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.066083 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.066755 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.068363 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.078708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1411/2798] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:37:57.082947 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.093229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9ce89-e53c-46a1-8083-68e1f370d8e5 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.098770 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.099367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.099647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.099912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.100553 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.102140 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd94df-3bc7-4b2c-8600-0d6682da94af admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.112802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1388/2799] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:57.117078 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.127953 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53cb8d-d632-4a77-9a17-1f83e36223f4 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.132892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.133549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.133830 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.134126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] PATH_INFO: `/v3/projects/92a1ba4aac44478b832e7ab0d9c7c3d4/users/2d3970a85c1944d3921975632c34a270/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.149145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a1ba4aac44478b832e7ab0d9c7c3d4, user_id=2d3970a85c1944d3921975632c34a270, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.151844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.166468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1412/2800] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:57 2022] PUT /identity/v3/projects/92a1ba4aac44478b832e7ab0d9c7c3d4/users/2d3970a85c1944d3921975632c34a270/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:57.177028 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.177312 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.177583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.207285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] MFA Rules not processed for user `2d3970a85c1944d3921975632c34a270`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:57.208497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:57.283078 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1389/2801] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:57.296529 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.298581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:57.360799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aaa80a-e9e2-4da6-a8b1-7958e0943aeb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.365037 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.365649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.365932 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.366201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.375162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.376606 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f6f2f-a1f9-4716-83eb-33fa951dc5e6 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.385561 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1413/2802] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:57.410893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.452616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:57.477655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be8f39-be24-41c9-b64e-0aa1d8ea2f90 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.483151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.483786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.484070 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.484342 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] PATH_INFO: `/v3/users/2d3970a85c1944d3921975632c34a270` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.485576 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3970a85c1944d3921975632c34a270)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.487566 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.537414 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3970a85c1944d3921975632c34a270'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:57.545453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3970a85c1944d3921975632c34a270'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:57.557780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3970a85c1944d3921975632c34a270'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:57.558450 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3970a85c1944d3921975632c34a270'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:57.566356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3970a85c1944d3921975632c34a270'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:57.574917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1390/2803] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:37:57 2022] DELETE /identity/v3/users/2d3970a85c1944d3921975632c34a270 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:57.579654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919608d-b354-4023-8268-1168e89537b8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.579940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919608d-b354-4023-8268-1168e89537b8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.580211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919608d-b354-4023-8268-1168e89537b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.601004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2919608d-b354-4023-8268-1168e89537b8 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:57.602201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2919608d-b354-4023-8268-1168e89537b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:57.664413 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1414/2804] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:57.674742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:57.676444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:57.739824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0830a-27b8-4000-9f75-3163463d2695 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:57.743630 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:57.744252 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:57.744531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:57.744801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:57.754632 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:57.756083 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd22175-eea0-49c1-9265-24aa5d72998a service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:57.765459 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1391/2805] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:58.065198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919608d-b354-4023-8268-1168e89537b8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.066814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2919608d-b354-4023-8268-1168e89537b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:58.079358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919608d-b354-4023-8268-1168e89537b8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.084271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.084920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.085199 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.085499 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] PATH_INFO: `/v3/projects/92a1ba4aac44478b832e7ab0d9c7c3d4` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.086689 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a1ba4aac44478b832e7ab0d9c7c3d4)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.088613 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.099975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:58.140223 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a1ba4aac44478b832e7ab0d9c7c3d4', 'request_id': 'req-32220a79-375d-4d18-a249-3b37513af44f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:58.145761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a1ba4aac44478b832e7ab0d9c7c3d4', 'request_id': 'req-32220a79-375d-4d18-a249-3b37513af44f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:58.152029 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a1ba4aac44478b832e7ab0d9c7c3d4', 'request_id': 'req-32220a79-375d-4d18-a249-3b37513af44f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:58.166291 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] The token cache is being invalidate because project 92a1ba4aac44478b832e7ab0d9c7c3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:37:58.166603 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:58.167294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-32220a79-375d-4d18-a249-3b37513af44f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:37:58.168963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1415/2806] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:37:58 2022] DELETE /identity/v3/projects/92a1ba4aac44478b832e7ab0d9c7c3d4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:58.173935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.174419 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.174548 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.195055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:58.196267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:58.248293 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:58.258998 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1392/2807] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:58.267966 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.268253 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.268538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.289115 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:58.290360 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:58.300547 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1416/2808] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:58.304955 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.316304 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e0784-62d7-4f87-81ef-7c28ddcabb8a None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.322661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.323217 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.323486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.323762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.324405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:37:58.324786 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.326417 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.338239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1393/2809] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:37:58.342489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.351914 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786bcac-f72b-4be5-8904-2d418c99199a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.357053 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.357615 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.357893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.358162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.358834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.360737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.389686 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1417/2810] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:37:58 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:58.395963 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.408579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca3e4-82ad-4327-a2d9-50e44b67637d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.468943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.469606 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.469911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.470219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.471013 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.473160 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.495211 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1394/2811] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:37:58 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:37:58.499580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.509939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813e501-ecfa-4bb0-97b9-e9b6661c17eb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.514903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.515516 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.515801 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.516101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.516752 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.518392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.528683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1418/2812] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:58.532502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.542421 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e9e03-6be6-4180-9a1c-d072f54b8bbb admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.547260 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.547898 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.548147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.548426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.549097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.550744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635195ac-8d49-40fe-a226-4f8e7c66cb29 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.561666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1395/2813] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:37:58.566900 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.578193 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3ab8-3e66-4641-bd7b-376850c4bed1 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.583899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.584595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.584840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.585114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] PATH_INFO: `/v3/projects/4766bbe64b2b4230b297b6552ec7f3b5/users/84f81268409f497fb59e4d63b5b3f727/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.600493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4766bbe64b2b4230b297b6552ec7f3b5, user_id=84f81268409f497fb59e4d63b5b3f727, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.603331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.622635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1419/2814] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:37:58 2022] PUT /identity/v3/projects/4766bbe64b2b4230b297b6552ec7f3b5/users/84f81268409f497fb59e4d63b5b3f727/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:37:58.633821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.634113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.634549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.664854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] MFA Rules not processed for user `84f81268409f497fb59e4d63b5b3f727`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:37:58.666109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:58.735370 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1396/2815] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:37:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:37:58.746637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:58.748732 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:58.807668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b659c-381e-4e90-a64a-8af74c6aa7c0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:58.811722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:58.812335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:58.812618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:58.812889 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:58.821670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:58.823153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87133902-7a34-4064-8fd4-c667aac1cc11 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:58.832098 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1420/2816] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:38:08.732155 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:08.828112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:08.838802 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:08.903734 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7819e5-94c1-455d-b1c8-d07cdc26e670 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:08.908711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:08.909352 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:08.909627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:08.909896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] PATH_INFO: `/v3/users/84f81268409f497fb59e4d63b5b3f727` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:08.911071 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f81268409f497fb59e4d63b5b3f727)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:08.912989 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:08.968131 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f81268409f497fb59e4d63b5b3f727'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:08.976727 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f81268409f497fb59e4d63b5b3f727'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:08.989143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f81268409f497fb59e4d63b5b3f727'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:08.989795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f81268409f497fb59e4d63b5b3f727'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:08.996626 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f81268409f497fb59e4d63b5b3f727'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.002614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1397/2817] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:38:08 2022] DELETE /identity/v3/users/84f81268409f497fb59e4d63b5b3f727 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:38:09.007338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.007621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.007893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.027343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:09.028537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:09.089047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1421/2818] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:09.098999 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.100593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:09.161746 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b49b-82d5-403c-ad14-ae34f022d251 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.165863 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.166507 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.166783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.167050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.176543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.177983 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9635f4-f5d6-4069-bc31-72720657699b service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.187535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1398/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:38:09.462325 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.464073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:09.476342 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d9d99-eaf1-479c-bea7-de8586adf0cf None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.481356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.482040 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.482323 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.482626 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] PATH_INFO: `/v3/projects/4766bbe64b2b4230b297b6552ec7f3b5` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.483825 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4766bbe64b2b4230b297b6552ec7f3b5)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.485730 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.534454 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4766bbe64b2b4230b297b6552ec7f3b5', 'request_id': 'req-00549c56-6c27-48bf-a36d-b659060860a0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.546424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4766bbe64b2b4230b297b6552ec7f3b5', 'request_id': 'req-00549c56-6c27-48bf-a36d-b659060860a0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.553004 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4766bbe64b2b4230b297b6552ec7f3b5', 'request_id': 'req-00549c56-6c27-48bf-a36d-b659060860a0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.566833 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] The token cache is being invalidate because project 4766bbe64b2b4230b297b6552ec7f3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:38:09.567145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.567754 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00549c56-6c27-48bf-a36d-b659060860a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:09.569240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1422/2820] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:38:09 2022] DELETE /identity/v3/projects/4766bbe64b2b4230b297b6552ec7f3b5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:38:09.574171 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.574481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.574750 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.594805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:09.596077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:09.613164 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:09.657458 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1399/2821] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:38:09.665379 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.665663 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.665942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.687165 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:09.688386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:09.698539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1423/2822] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:09.703316 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.713371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44429d-2942-47b3-9c28-b6622b53ea75 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.719205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.719819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.720097 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.720385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.721201 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:38:09.721502 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.723186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.736255 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1400/2823] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:38:09.740961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.752831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ab1ea-1c18-4c58-91c5-8fbcb0e71824 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.759501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.760114 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.760395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.760640 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.761294 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.763292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.793867 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1424/2824] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:38:09 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:38:09.798073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.808147 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd960d-b401-4e3c-8763-75325cf76e42 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.864624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.865226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.865506 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.865796 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.866469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.868392 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.890948 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1401/2825] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:38:09 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:38:09.895280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.905936 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dad085-7399-4c04-911f-91ee78b299f5 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.911541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.912085 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.912361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.912627 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.913280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.914942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.924965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1425/2826] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:38:09.928962 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.940004 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30159f7e-e1ac-4b79-8670-702c3058467a admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.945499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.946107 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.946403 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.946683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.947336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:09.948943 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb044-dcee-443e-a175-0ddd97069710 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:09.959928 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1402/2827] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:38:09.964347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:09.974675 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e633e7-93e0-4db2-9043-404787f2d421 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:09.980214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:09.980866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:09.981145 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:09.981420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] PATH_INFO: `/v3/projects/2bd38a578e1e4764a41c5fcb32ca3cfc/users/27860df658d8416fb862d0eacff1169b/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:09.997566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd38a578e1e4764a41c5fcb32ca3cfc, user_id=27860df658d8416fb862d0eacff1169b, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:10.000271 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:10.015168 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1426/2828] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:38:09 2022] PUT /identity/v3/projects/2bd38a578e1e4764a41c5fcb32ca3cfc/users/27860df658d8416fb862d0eacff1169b/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:38:10.026945 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a92-54e0-4a6e-83bb-30079f3f2ee3 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:10.027245 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a92-54e0-4a6e-83bb-30079f3f2ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:10.027514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497a92-54e0-4a6e-83bb-30079f3f2ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:10.058698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f6497a92-54e0-4a6e-83bb-30079f3f2ee3 None None] MFA Rules not processed for user `27860df658d8416fb862d0eacff1169b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:10.059912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f6497a92-54e0-4a6e-83bb-30079f3f2ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:10.140309 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1403/2829] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:10.151570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:10.153431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:10.177398 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:10.213260 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831ff7-2454-44c2-bc59-eeafcbd397eb admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:10.217265 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:10.217892 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:10.218169 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:10.218449 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:10.227307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:10.228852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4812615-2566-400c-b5b3-c3eed3413025 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:10.238008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1427/2830] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:38:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:38:11.627269 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:11.627821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:11.627821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:11.648111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] MFA Rules not processed for user `27860df658d8416fb862d0eacff1169b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:11.649311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:11.657881 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1404/2831] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:20.233879 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=17 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:21.641977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:25.478772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20642e3d-811b-46b9-aa98-eae8a58ea44e None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:25.479252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20642e3d-811b-46b9-aa98-eae8a58ea44e None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:25.479348 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20642e3d-811b-46b9-aa98-eae8a58ea44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:25.540097 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-20642e3d-811b-46b9-aa98-eae8a58ea44e None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:25.541327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-20642e3d-811b-46b9-aa98-eae8a58ea44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:25.607092 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1428/2832] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:38:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:25.656685 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:25.658290 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:25.754832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b179c-1d89-463a-aecb-fc01ddcfeb09 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:25.760288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:25.760911 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:25.761192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:25.761471 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:25.769832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:25.772157 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:25.780438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1405/2833] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:38:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:38:26.327415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda548f-f7ad-4ced-a204-2bb50095e688 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:26.327642 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda548f-f7ad-4ced-a204-2bb50095e688 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:26.327910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda548f-f7ad-4ced-a204-2bb50095e688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:26.348049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-afda548f-f7ad-4ced-a204-2bb50095e688 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:26.349262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-afda548f-f7ad-4ced-a204-2bb50095e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:26.357258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1429/2834] 10.222.0.52 () {58 vars in 967 bytes} [Tue Sep 13 20:38:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:26.370474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:26.381717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01c8bc-2868-480d-a73f-2ab4c965be81 service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:26.387697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:26.388266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:26.388538 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:26.388805 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:26.397890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:26.399503 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:26.408088 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1406/2835] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:38:26.643167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede95e24-57d9-48b1-96bc-b917f638ff34 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:26.643628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede95e24-57d9-48b1-96bc-b917f638ff34 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:26.643731 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede95e24-57d9-48b1-96bc-b917f638ff34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:26.664277 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ede95e24-57d9-48b1-96bc-b917f638ff34 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:26.665493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ede95e24-57d9-48b1-96bc-b917f638ff34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:26.673851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1430/2836] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Sep 13 20:38:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:26.686205 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:26.698310 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe6b5-d5a8-4f8e-972d-4f67473663a9 service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:26.704792 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:26.705395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:26.706558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:26.706839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:26.716523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:26.718109 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:26.726920 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1407/2837] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:38:27.064315 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:27.064628 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:27.064901 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:27.087067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] MFA Rules not processed for user `398ef52b1f264cab85425daaa03e2b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:27.088197 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:27.099453 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1431/2838] 10.222.0.52 () {58 vars in 967 bytes} [Tue Sep 13 20:38:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:27.117958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:27.130280 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aada6-84c8-4326-bb57-87b87f649faf service swift] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:27.137149 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09cae6567bc540bab88dc59c1ddee27b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:27.137753 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:27.138032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:27.138303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:27.147366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:27.149075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9dd6a-f279-4295-963d-43780cce4064 service swift] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:27.157497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1408/2839] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Sep 13 20:38:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:38:29.446949 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:29.511031 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8826a89-4957-4921-a358-73956f7715f9 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:29.515753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:29.516408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:29.516683 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:29.516957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] PATH_INFO: `/v3/users/27860df658d8416fb862d0eacff1169b` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:29.518091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27860df658d8416fb862d0eacff1169b)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:29.520060 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:29.570886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:29.572457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27860df658d8416fb862d0eacff1169b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:29.580815 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27860df658d8416fb862d0eacff1169b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:29.592447 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27860df658d8416fb862d0eacff1169b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:29.593077 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27860df658d8416fb862d0eacff1169b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:29.599736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27860df658d8416fb862d0eacff1169b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:29.605576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1432/2840] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:38:29 2022] DELETE /identity/v3/users/27860df658d8416fb862d0eacff1169b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:38:29.610113 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:29.610740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:29.610740 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:29.629428 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:29.630653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:29.691715 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1409/2841] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:38:29.701729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:29.703386 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:29.769865 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96616f-c781-4bb9-8e69-35cf927848e7 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:29.773916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:29.774772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:29.775198 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:29.775659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:29.786136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:29.787659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:29.797524 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1433/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:38:29.971653 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:29.973242 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:29.986473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fc2f-de37-4d79-ba11-3f56e5aa9163 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:29.991683 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:29.992335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:29.992612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:29.992885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] PATH_INFO: `/v3/projects/2bd38a578e1e4764a41c5fcb32ca3cfc` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:29.994525 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd38a578e1e4764a41c5fcb32ca3cfc)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:30.001760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:30.034395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:30.051657 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd38a578e1e4764a41c5fcb32ca3cfc', 'request_id': 'req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:30.057837 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd38a578e1e4764a41c5fcb32ca3cfc', 'request_id': 'req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:30.064659 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd38a578e1e4764a41c5fcb32ca3cfc', 'request_id': 'req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:30.077777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] The token cache is being invalidate because project 2bd38a578e1e4764a41c5fcb32ca3cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:38:30.078059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:30.078769 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-e553b50b-ea7b-48d6-ad6c-ded784f4f56f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:38:30.080366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1410/2843] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:38:29 2022] DELETE /identity/v3/projects/2bd38a578e1e4764a41c5fcb32ca3cfc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:38:39.793214 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=39,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:40.074663 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:40.091283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:38:40.093971 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:40.187631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11af01-8554-455e-97bd-aeead9eda9da service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:38:40.268835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:38:40.269481 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:40.269761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:40.270034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:40.271987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:40.286665 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:38:40.288282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4766eb-e762-4c88-8b8b-b906d8dc772e service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:38:40.300649 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1434/2844] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:38:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:38:43.249215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:38:43.249919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:38:43.249919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:38:43.270707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] MFA Rules not processed for user `6c42757a041b4831abd7a7ba66cbe933`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:38:43.272055 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:38:43.333738 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1411/2845] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:38:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:38:50.295998 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=62 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:38:53.324577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=25 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:26.897438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:26.897997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:26.897997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:26.917964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] MFA Rules not processed for user `d20ba942c80848d7819aede8c0df1e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:26.919207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:26.998136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1435/2846] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:36.990964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=31 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:54.738968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:54.740432 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:54.864991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdd833-9b87-4df5-9055-d2fef53bd515 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:54.869624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:54.870256 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:54.870568 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:54.870876 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] PATH_INFO: `/v3/users/6c42757a041b4831abd7a7ba66cbe933` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:54.872062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c42757a041b4831abd7a7ba66cbe933)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:54.873991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:54.929559 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c42757a041b4831abd7a7ba66cbe933'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:54.938674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c42757a041b4831abd7a7ba66cbe933'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:54.950814 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c42757a041b4831abd7a7ba66cbe933'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:54.951441 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c42757a041b4831abd7a7ba66cbe933'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:54.959085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-98f1998c-2216-4c78-aa61-bf6c4037d233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c42757a041b4831abd7a7ba66cbe933'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:54.965382 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1412/2847] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:39:54 2022] DELETE /identity/v3/users/6c42757a041b4831abd7a7ba66cbe933 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:55.145252 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.179245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574be10-f53a-458d-934e-ddd67b14388f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.238929 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.239583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.239863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.240134 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] PATH_INFO: `/v3/projects/be0640c4e0584998a646e45565347687` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.241737 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0640c4e0584998a646e45565347687)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.243690 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.291404 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0640c4e0584998a646e45565347687', 'request_id': 'req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:55.297659 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0640c4e0584998a646e45565347687', 'request_id': 'req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:55.304673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0640c4e0584998a646e45565347687', 'request_id': 'req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:55.317995 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] The token cache is being invalidate because project be0640c4e0584998a646e45565347687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:39:55.318289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:55.318973 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-0489e9d7-8f71-48c6-bb5a-1fc976841c9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:55.320494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1436/2848] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:39:55 2022] DELETE /identity/v3/projects/be0640c4e0584998a646e45565347687 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:55.326232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.326232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.326232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.365365 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:55.388603 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:55.389745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:55.449133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1413/2849] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:55.457728 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.458039 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.458285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.478416 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:55.479653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:55.490959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1437/2850] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:55.492162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.503111 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203236e-1fbb-4ab9-97df-730ba60f92d5 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.507935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.508558 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.508840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.509117 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.509774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:39:55.510079 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.511751 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.527159 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1414/2851] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:39:55.528318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.537662 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306d13d-09a9-4c7b-a439-5a46980ff9e5 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.542444 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.543068 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.543356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.543629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.544269 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.546216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.586570 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1438/2852] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:39:55 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:39:55.590993 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.602166 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6315fb-392d-4b86-9cbc-a81d7ede8835 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.658412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.659001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.659262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.659511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.660173 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.662101 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.685745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1415/2853] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:39:55 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:39:55.690696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.705456 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbf3d-4838-4f89-82aa-a796591084d3 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.710886 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.711592 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.711939 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.712282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.713007 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.714766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.729326 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1439/2854] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:39:55.730652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.740929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a754a0-2f34-41d3-bd98-bc63b0886f8d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.745875 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.746509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.746795 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.747072 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.747723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.749335 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d5d34-1a82-4b62-8c69-8e821c602513 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.760016 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1416/2855] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:39:55.764190 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.774263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec3211-0223-4cd5-b5a6-4614555b3033 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:55.779149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:55.779818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.780091 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.780358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] PATH_INFO: `/v3/projects/a05f0392090f485a8927ee077c554610/users/41600aac56ab4e9ba0edd529ab9042d3/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.796256 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05f0392090f485a8927ee077c554610, user_id=41600aac56ab4e9ba0edd529ab9042d3, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:55.798997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:55.815826 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1440/2856] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:39:55 2022] PUT /identity/v3/projects/a05f0392090f485a8927ee077c554610/users/41600aac56ab4e9ba0edd529ab9042d3/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:55.826771 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:55.827161 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:55.827554 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:55.857978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] MFA Rules not processed for user `41600aac56ab4e9ba0edd529ab9042d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:55.859243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:55.907691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:55.930337 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1417/2857] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:55.942721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:55.944661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:55.963947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:56.057743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a0bc6-8281-4a32-b902-532f549f7283 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:56.065102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463fcb701698473bad12e8101b9018ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:56.065790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:56.066070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:56.066330 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:56.074733 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:56.076335 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6e4c2-f765-4652-825e-b7073f62d668 service nova] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:56.085952 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1441/2858] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:39:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:39:56.290492 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:56.359005 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86431-fa59-4b2c-9089-b8c2e6244a66 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:56.363916 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:56.364494 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:56.364772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:56.365063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] PATH_INFO: `/v3/users/41600aac56ab4e9ba0edd529ab9042d3` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:56.366219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=41600aac56ab4e9ba0edd529ab9042d3)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:56.368170 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:56.427677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41600aac56ab4e9ba0edd529ab9042d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:56.436531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41600aac56ab4e9ba0edd529ab9042d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:56.448906 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41600aac56ab4e9ba0edd529ab9042d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:56.449561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41600aac56ab4e9ba0edd529ab9042d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:56.456747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41600aac56ab4e9ba0edd529ab9042d3'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:56.462968 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1418/2859] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:39:56 2022] DELETE /identity/v3/users/41600aac56ab4e9ba0edd529ab9042d3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:56.467556 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:56.468035 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:56.468124 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:56.487238 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:56.488438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:56.548567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1442/2860] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:56.558745 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:56.560541 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:56.637880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddca7-6607-40f1-b4ff-6d2c8e1aa2ce admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:56.644438 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:56.645267 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:56.645569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:56.645838 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:56.659130 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:56.661281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc896108-0cb5-4937-bd8d-7ac0d945e635 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:56.675556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1419/2861] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:39:56.952885 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:56.954396 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:56.967944 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d0acb-1366-4c81-8597-f3df0b6a645a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:56.973317 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:56.973994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:56.974298 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:56.974609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] PATH_INFO: `/v3/projects/a05f0392090f485a8927ee077c554610` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:56.975707 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05f0392090f485a8927ee077c554610)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:56.977635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.027682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05f0392090f485a8927ee077c554610', 'request_id': 'req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:57.033562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05f0392090f485a8927ee077c554610', 'request_id': 'req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:57.035484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:57.040425 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05f0392090f485a8927ee077c554610', 'request_id': 'req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:57.053475 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] The token cache is being invalidate because project a05f0392090f485a8927ee077c554610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:39:57.053762 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:57.054525 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-40a3a1b9-4ccf-4f63-a116-fd1d97f12dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:39:57.056121 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1443/2862] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:39:56 2022] DELETE /identity/v3/projects/a05f0392090f485a8927ee077c554610 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:57.061253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.062069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.062190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.071128 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:57.083022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:57.084820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:57.146481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1420/2863] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:57.154942 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.155227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.155515 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.176179 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:57.177357 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:57.186008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1444/2864] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:57.191090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.201586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518602a-56a9-4ab1-bfaa-99cad1eb7b70 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.207522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.208137 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.208422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.208701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.209358 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:39:57.209661 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.211311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.223129 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1421/2865] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:39:57.227705 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.237654 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06709e64-c6b9-4f2e-929e-e50e92b3c030 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.243609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.244212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.244486 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.244997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.245634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.247577 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.279222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1445/2866] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:39:57.283692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.294023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b42633-5be6-49d0-9a47-3e76767f819b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.351012 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.351616 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.351890 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.352156 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.352774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.354736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.378162 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1422/2867] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:39:57.384235 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.395523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe660-539c-449a-ba40-b1593cbdf775 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.401364 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.401984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.402275 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.402573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.403218 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.404844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.415376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1446/2868] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:39:57.419569 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.430096 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f898bbc-8cc8-4c3b-9883-6054d61f3b3b admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.435426 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.436032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.436308 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.436579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.437578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.439207 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc57f25-fc37-476a-adaf-099c4ffc1b5a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.449908 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1423/2869] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:39:57.454022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.464362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab681043-f863-4a4d-8d8c-0de91ccd25aa admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.469331 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.469978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.470262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.470576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] PATH_INFO: `/v3/projects/7b2a44185caf49269752e6a2802e3685/users/e5c32a7b73c441ff9f33f5a8e9b3d76b/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.484994 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2a44185caf49269752e6a2802e3685, user_id=e5c32a7b73c441ff9f33f5a8e9b3d76b, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.487899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.506189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1447/2870] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:39:57 2022] PUT /identity/v3/projects/7b2a44185caf49269752e6a2802e3685/users/e5c32a7b73c441ff9f33f5a8e9b3d76b/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:57.520179 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.520511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.520869 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.559239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] MFA Rules not processed for user `e5c32a7b73c441ff9f33f5a8e9b3d76b`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:57.561051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:57.627353 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:57.635562 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1424/2871] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:57.636800 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.701888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d778f-9028-459a-bda6-152fb2a26acd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.707257 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.707802 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.708081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.708362 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.708991 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.710951 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.739248 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1448/2872] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:39:57.743564 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.754919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16782c-0716-4a18-87ab-f87ba26f89cd None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.813531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.814143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.814444 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.814717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.815416 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.817338 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.839997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1425/2873] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:39:57 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:39:57.844106 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.854681 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3f36b-b318-43c1-b873-8ef0d93be159 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.859788 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.860392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.860674 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.860945 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.861589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.863276 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.874468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1449/2874] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:39:57.878820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.889199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d960d8e-2a9d-44a4-a248-daa23cc40bc2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:57.894974 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:57.895652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:57.895941 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:57.896219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] PATH_INFO: `/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c/users/0c98f4ff013e4e67940af78003caf63c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:57.914065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0501a69cb691480d8ce80c2eb88d7b1c, user_id=0c98f4ff013e4e67940af78003caf63c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:57.916787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:57.931262 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1426/2875] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:39:57 2022] PUT /identity/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c/users/0c98f4ff013e4e67940af78003caf63c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:57.935289 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:57.945539 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33bb5-f9d2-4426-9a06-525919101f2a admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:58.003420 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:58.004076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:58.004399 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:58.004677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:58.005327 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:58.006983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7bee-ce6b-4dbb-bf42-e90f1a097f7b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:58.008282 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:39:58.018624 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1450/2876] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:39:58.022867 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:58.033169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4d1b8-c584-46ec-bd5a-cab027bc1953 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:58.038263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:58.038958 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:58.039239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:58.039511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] PATH_INFO: `/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c/users/0c98f4ff013e4e67940af78003caf63c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:58.041581 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0501a69cb691480d8ce80c2eb88d7b1c, user_id=0c98f4ff013e4e67940af78003caf63c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:58.044273 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:58.054940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1427/2877] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:39:58 2022] PUT /identity/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c/users/0c98f4ff013e4e67940af78003caf63c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:39:58.065777 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:58.066054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:58.066329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:58.095201 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] MFA Rules not processed for user `0c98f4ff013e4e67940af78003caf63c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:39:58.096809 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:58.166285 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1451/2878] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:39:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:39:58.178186 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:58.180069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:39:58.239752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f671e-0bf6-4725-bbac-3f2b3d950912 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:58.243756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:58.244366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:58.244640 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:58.244907 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:58.254617 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:58.256042 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:58.266752 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1428/2879] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:39:58.346948 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:39:58.357542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946e55-a46b-4e89-9df0-1c852a0f6783 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:39:58.361913 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:39:58.362557 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:39:58.362832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:39:58.363104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:39:58.423385 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:39:58.424877 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f364201-0dcc-4a9f-9d63-3967166052bc service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:39:58.434306 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1452/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:39:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:08.261754 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:08.430033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=64 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:08.640726 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:08.705547 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50fa12-f6d7-4847-b542-1c4174b3ec7d service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:08.711681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:08.712474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:08.712801 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:08.713049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] PATH_INFO: `/v3/users/e5c32a7b73c441ff9f33f5a8e9b3d76b` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:08.714522 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c32a7b73c441ff9f33f5a8e9b3d76b)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:08.716485 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:08.763604 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c32a7b73c441ff9f33f5a8e9b3d76b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:08.771965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c32a7b73c441ff9f33f5a8e9b3d76b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:08.783879 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c32a7b73c441ff9f33f5a8e9b3d76b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:08.784595 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c32a7b73c441ff9f33f5a8e9b3d76b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:08.792445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c32a7b73c441ff9f33f5a8e9b3d76b'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:08.799313 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1429/2881] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:08 2022] DELETE /identity/v3/users/e5c32a7b73c441ff9f33f5a8e9b3d76b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:08.804013 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:08.804272 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:08.804541 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:08.825554 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:08.827075 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:08.887538 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1453/2882] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:08.897724 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:08.899501 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:08.960832 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c4db8-734a-4582-a3ff-6f964f7e861c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:08.964787 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:08.965393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:08.965670 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:08.965935 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:08.975363 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:08.976840 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:08.987067 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1430/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:09.259634 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:09.261131 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:09.277153 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5095cd7-7f83-4d49-8a9a-37780c25d19c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:09.282232 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:09.282903 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:09.283183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:09.283448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] PATH_INFO: `/v3/projects/7b2a44185caf49269752e6a2802e3685` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:09.284580 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2a44185caf49269752e6a2802e3685)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:09.286553 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:09.335933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2a44185caf49269752e6a2802e3685', 'request_id': 'req-5e06f973-a073-49a4-93ec-4c9bd78570ac'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.341587 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2a44185caf49269752e6a2802e3685', 'request_id': 'req-5e06f973-a073-49a4-93ec-4c9bd78570ac'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.347682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2a44185caf49269752e6a2802e3685', 'request_id': 'req-5e06f973-a073-49a4-93ec-4c9bd78570ac'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.361105 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] The token cache is being invalidate because project 7b2a44185caf49269752e6a2802e3685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:09.361387 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.361997 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.363474 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1454/2884] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:09 2022] DELETE /identity/v3/projects/7b2a44185caf49269752e6a2802e3685 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:09.367697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:09.377896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7441d-c949-40b4-9a68-e6690c6a5af0 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:09.397788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:09.438675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:09.439314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:09.439624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:09.439899 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] PATH_INFO: `/v3/users/0c98f4ff013e4e67940af78003caf63c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:09.441068 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c98f4ff013e4e67940af78003caf63c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:09.443049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:09.492311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c98f4ff013e4e67940af78003caf63c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.499314 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c98f4ff013e4e67940af78003caf63c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.511230 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c98f4ff013e4e67940af78003caf63c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.511931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c98f4ff013e4e67940af78003caf63c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.519711 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-9ec154f9-adff-4398-ae3b-f77df334046a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c98f4ff013e4e67940af78003caf63c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.526144 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1431/2885] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:09 2022] DELETE /identity/v3/users/0c98f4ff013e4e67940af78003caf63c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:09.804986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:09.806703 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:09.828213 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06f973-a073-49a4-93ec-4c9bd78570ac admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:09.886080 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:09.886682 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:09.886963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:09.887237 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] PATH_INFO: `/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:09.888353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0501a69cb691480d8ce80c2eb88d7b1c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:09.890283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:09.922834 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:09.940054 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0501a69cb691480d8ce80c2eb88d7b1c', 'request_id': 'req-4dcada93-62fa-4dfb-8554-b8f31e9afb44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.945434 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0501a69cb691480d8ce80c2eb88d7b1c', 'request_id': 'req-4dcada93-62fa-4dfb-8554-b8f31e9afb44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.951709 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0501a69cb691480d8ce80c2eb88d7b1c', 'request_id': 'req-4dcada93-62fa-4dfb-8554-b8f31e9afb44'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.965207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] The token cache is being invalidate because project 0501a69cb691480d8ce80c2eb88d7b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:09.965494 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.966112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-4dcada93-62fa-4dfb-8554-b8f31e9afb44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:09.967607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1455/2886] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:09 2022] DELETE /identity/v3/projects/0501a69cb691480d8ce80c2eb88d7b1c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:09.972602 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:09.972895 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:09.974183 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:09.994247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:09.995523 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:10.060800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1432/2887] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:10.070488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.070768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.071034 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.091196 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:10.092378 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:10.100767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1456/2888] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:10.105108 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.115545 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5069ca-cca1-4873-b7d9-fb9fb8356394 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.121058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.121668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.121956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.122229 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.122925 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:40:10.123232 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.124829 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.137679 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1433/2889] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:40:10.142302 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.151846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c61cd-cff8-45ed-bbb8-c675d044d19a None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.157050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.157661 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.157941 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.158216 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.158895 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.160798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.190225 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1457/2890] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:40:10.194651 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.204788 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2dd19-8952-44c2-bab6-8d4b9f37b26c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.230857 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:10.265271 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.265885 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.266168 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.266468 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.267100 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.269049 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.290793 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1434/2891] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:10.294912 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.305595 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d333-b529-486e-a1c3-cd8554dee772 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.311729 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.312365 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.312635 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.312881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.313523 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.315376 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.325631 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1458/2892] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:10.329479 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.339791 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf4aaa-dd25-4f14-a132-0bb562c23ee6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.344919 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.345514 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.345790 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.346058 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.346728 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.348350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ddba3-072a-4c3c-a754-7020b0647bda admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.358235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1435/2893] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:10.362415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.372359 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c581abc-4cd2-460f-a609-7fc71edf545f admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.377432 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.378070 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.378344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.378599 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] PATH_INFO: `/v3/projects/f192c11a49344303b07deeed130ce335/users/23db3861647f45b2848b58efb0b8b0b1/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.393450 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f192c11a49344303b07deeed130ce335, user_id=23db3861647f45b2848b58efb0b8b0b1, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.396062 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.410770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1459/2894] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:10 2022] PUT /identity/v3/projects/f192c11a49344303b07deeed130ce335/users/23db3861647f45b2848b58efb0b8b0b1/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:40:10.422089 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.422636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.422636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.453218 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] MFA Rules not processed for user `23db3861647f45b2848b58efb0b8b0b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:10.454472 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:10.520862 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1436/2895] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:10.525835 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.594300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9548d-b401-44e9-81c5-8fe8b8231edd admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.600058 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.600653 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.600937 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.601211 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.601846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.603852 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.637543 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1460/2896] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:10.641985 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.653244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f3e96-cc34-4f48-8ec8-1ee78f1d7444 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.718827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.719447 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.719756 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.719984 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.720624 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.722593 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.747703 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1437/2897] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:10.752006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.763087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d231e2-ac09-476a-9193-b5b86dcaeda2 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.768581 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.769194 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.769473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.769779 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.770408 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.772043 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.783096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1461/2898] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:10.787423 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.798253 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9150b-f146-4a33-afee-b3e3d27ed360 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.804219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.804850 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.805143 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.805412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] PATH_INFO: `/v3/projects/afe9b9ac0b824c72be14c43969e12306/users/21bd9a3fea734a15b04f9ae9b78335d1/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.821393 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe9b9ac0b824c72be14c43969e12306, user_id=21bd9a3fea734a15b04f9ae9b78335d1, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.824116 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.840706 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1438/2899] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:10 2022] PUT /identity/v3/projects/afe9b9ac0b824c72be14c43969e12306/users/21bd9a3fea734a15b04f9ae9b78335d1/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:10.844832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.854924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81bde3-32b1-429e-811a-79b7a6d34c87 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.874431 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:10.913924 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.914488 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.914763 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.915099 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.915712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.917332 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e2576-f8c4-420a-85de-32995d284195 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.927329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1462/2900] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:10.931995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:10.937871 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:10.944691 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c03e8-3164-4ef1-a110-a09129f91c5f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:10.950051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:10.950742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.951026 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.951300 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] PATH_INFO: `/v3/projects/afe9b9ac0b824c72be14c43969e12306/users/21bd9a3fea734a15b04f9ae9b78335d1/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:10.953294 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe9b9ac0b824c72be14c43969e12306, user_id=21bd9a3fea734a15b04f9ae9b78335d1, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:10.956022 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:10.966243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1439/2901] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:10 2022] PUT /identity/v3/projects/afe9b9ac0b824c72be14c43969e12306/users/21bd9a3fea734a15b04f9ae9b78335d1/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:10.976542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:10.976831 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:10.977103 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.005314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] MFA Rules not processed for user `21bd9a3fea734a15b04f9ae9b78335d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:11.006571 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:11.077910 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1463/2902] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:11.094427 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:11.096971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:11.157698 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741820a6-0cc3-4382-bdba-cd6e77336198 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:11.161395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:11.162003 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:11.162281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:11.162589 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.171643 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:11.173033 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a752b8-83a1-46bf-9a9e-a719e41223e5 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:11.182442 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1440/2903] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:11.249896 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:11.312810 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5d3d7-7f70-4887-adbe-4684dc04ccd8 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:11.322869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:11.323959 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:11.324361 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:11.324795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] PATH_INFO: `/v3/users/23db3861647f45b2848b58efb0b8b0b1` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.326743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23db3861647f45b2848b58efb0b8b0b1)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:11.329947 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:11.381307 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23db3861647f45b2848b58efb0b8b0b1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.382989 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:11.391006 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23db3861647f45b2848b58efb0b8b0b1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.403424 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23db3861647f45b2848b58efb0b8b0b1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.404072 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23db3861647f45b2848b58efb0b8b0b1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.410938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23db3861647f45b2848b58efb0b8b0b1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.417870 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1464/2904] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:11 2022] DELETE /identity/v3/users/23db3861647f45b2848b58efb0b8b0b1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:11.422611 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:11.423065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:11.423190 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.442486 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:11.443690 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:11.504062 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1441/2905] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:40:11.514096 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:11.515772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:11.576576 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e478-55b7-44da-a5be-d647604df1e6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:11.581304 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:11.581919 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:11.582228 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:11.582511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.592300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:11.593745 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:11.603207 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1465/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:11.882912 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:11.884676 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:11.896621 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450c283-dbdc-485a-bbed-df42cf4ea7be None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:11.901777 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:11.902605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:11.902929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:11.903192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] PATH_INFO: `/v3/projects/f192c11a49344303b07deeed130ce335` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:11.905095 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f192c11a49344303b07deeed130ce335)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:11.907102 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:11.956150 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f192c11a49344303b07deeed130ce335', 'request_id': 'req-7ae26db3-52c2-4400-8a26-917e706be1c1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.961992 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f192c11a49344303b07deeed130ce335', 'request_id': 'req-7ae26db3-52c2-4400-8a26-917e706be1c1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.968249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f192c11a49344303b07deeed130ce335', 'request_id': 'req-7ae26db3-52c2-4400-8a26-917e706be1c1'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.982123 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] The token cache is being invalidate because project f192c11a49344303b07deeed130ce335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:11.982436 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.983053 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:11.984531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1442/2907] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:11 2022] DELETE /identity/v3/projects/f192c11a49344303b07deeed130ce335 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:11.988753 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:11.998846 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e616c4c-7776-4882-9ab3-71887b429cfb service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.057240 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.057875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.058148 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.058469 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] PATH_INFO: `/v3/users/21bd9a3fea734a15b04f9ae9b78335d1` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.059605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21bd9a3fea734a15b04f9ae9b78335d1)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.061975 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.109392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21bd9a3fea734a15b04f9ae9b78335d1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.116986 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21bd9a3fea734a15b04f9ae9b78335d1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.127972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:12.129219 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21bd9a3fea734a15b04f9ae9b78335d1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.130126 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21bd9a3fea734a15b04f9ae9b78335d1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.141748 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31cafe30-4916-430a-b4c2-ef5f1ea93e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21bd9a3fea734a15b04f9ae9b78335d1'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.147963 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1466/2908] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:11 2022] DELETE /identity/v3/users/21bd9a3fea734a15b04f9ae9b78335d1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:12.432978 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.434583 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:12.450043 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae26db3-52c2-4400-8a26-917e706be1c1 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.463596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:12.514086 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.514642 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.514952 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.515220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] PATH_INFO: `/v3/projects/afe9b9ac0b824c72be14c43969e12306` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.516357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe9b9ac0b824c72be14c43969e12306)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.518325 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.571722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe9b9ac0b824c72be14c43969e12306', 'request_id': 'req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.577439 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe9b9ac0b824c72be14c43969e12306', 'request_id': 'req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.583695 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe9b9ac0b824c72be14c43969e12306', 'request_id': 'req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.597934 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] The token cache is being invalidate because project afe9b9ac0b824c72be14c43969e12306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:12.598225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.598901 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-af0afc61-f8f3-40f1-a8a0-74dad4cf7a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:12.600431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1443/2909] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:12 2022] DELETE /identity/v3/projects/afe9b9ac0b824c72be14c43969e12306 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:12.606798 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.607166 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.607445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.628074 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:12.629287 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:12.699677 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1467/2910] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:12.707481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae611-9950-427a-863e-aac3916b3394 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.707742 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae611-9950-427a-863e-aac3916b3394 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.708069 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae611-9950-427a-863e-aac3916b3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.729303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-413ae611-9950-427a-863e-aac3916b3394 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:12.730577 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-413ae611-9950-427a-863e-aac3916b3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:12.739682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1444/2911] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:12.744167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.754406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee4ecb-9c4a-49c9-98bc-97266983ec57 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.759669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.761088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.761088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.761088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] PATH_INFO: `/v3/domains` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.761410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:40:12.761721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.763356 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.776799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1468/2912] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:40:12.782531 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae611-9950-427a-863e-aac3916b3394 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.795487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae611-9950-427a-863e-aac3916b3394 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.800677 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.801299 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.801579 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.801854 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.802544 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.804474 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.835666 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1445/2913] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:12 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:12.840209 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.851274 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736a3e-99f2-4cb1-8e3f-8311a5cfd47e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.913133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.913744 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.914024 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.914292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.914982 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.916984 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.942878 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1469/2914] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:12 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:12.944244 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.954929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de715d9f-30d8-49e9-b7ca-e25f59b9977f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.960490 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.961090 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.961371 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.961748 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.962321 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.963956 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:12.974075 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1446/2915] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:12.977961 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:12.988270 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad5082-0fd6-4493-a1aa-40a82902a998 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:12.993699 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:12.994310 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:12.994605 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:12.994866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:12.995509 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:12.997112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db032a0-a101-4b23-8191-28578a293b61 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.008751 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1470/2916] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:13.013431 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.028237 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d3ec88-cc6e-4c84-b6a8-9f2c804faee8 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.033239 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.033893 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.034172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.034477 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] PATH_INFO: `/v3/projects/4129293efd83403887c1dd3d8975969a/users/655c253187444b428b941157118c6a7f/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.049821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4129293efd83403887c1dd3d8975969a, user_id=655c253187444b428b941157118c6a7f, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.052596 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.067865 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1447/2917] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:13 2022] PUT /identity/v3/projects/4129293efd83403887c1dd3d8975969a/users/655c253187444b428b941157118c6a7f/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:13.078517 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.078806 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.079076 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.110848 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] MFA Rules not processed for user `655c253187444b428b941157118c6a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:13.112081 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:13.157290 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:13.183636 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1471/2918] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:13.188692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.248136 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:13.272357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a75016-e4e1-4fb9-9165-4bc09ce3ce36 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.280578 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.281320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.281668 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.282029 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.283065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.285973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.318385 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1448/2919] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:40:13.322629 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.333926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176315f4-1797-4e3b-8715-56bab70bf7f2 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.391978 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.392582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.392855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.393118 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.393741 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.395692 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.419647 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1472/2920] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:13.423896 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.435644 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61411f-5e66-4d00-8225-72e17254023e admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.442614 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.443241 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.443524 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.443800 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.444440 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.446261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.458106 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1449/2921] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:13.464477 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.477542 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab07fde-bc4a-4c5d-bea6-fc935537724e admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.483670 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.484283 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.484562 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.484832 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.485473 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.487133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5b75e-c1ce-4210-8c1d-f663e900846b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.497807 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1473/2922] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:13.502341 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.517638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83075bdb-eda7-43bd-8972-b6980bf4c5ed admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.525351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.526031 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.526281 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.526591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] PATH_INFO: `/v3/projects/a7a9368a9f5c4e808dac4785f8cd75a0/users/b2852f3b1b8b450fabb10123f4ecd680/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.542032 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a9368a9f5c4e808dac4785f8cd75a0, user_id=b2852f3b1b8b450fabb10123f4ecd680, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.544660 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.560050 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1450/2923] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:13 2022] PUT /identity/v3/projects/a7a9368a9f5c4e808dac4785f8cd75a0/users/b2852f3b1b8b450fabb10123f4ecd680/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:13.571495 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.571783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.572468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.601061 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] MFA Rules not processed for user `b2852f3b1b8b450fabb10123f4ecd680`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:13.602263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:13.670993 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1474/2924] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:13.675755 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.738605 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aab25-907a-4e1a-9213-f01dbc449f63 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.744063 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.744669 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.744946 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.745231 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.745852 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.747816 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.778133 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1451/2925] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:13.782292 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.793712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09b419-a057-4eb7-bc08-391d21bc343c None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.841414 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:13.852876 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.853489 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.853766 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.854033 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.854700 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.856620 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.883087 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1475/2926] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:13 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:13.887396 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.897982 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed7d0-e5ee-42f6-9b9a-03fe8d0cca7c admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.903061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.903652 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.903929 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.904220 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.904877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.906499 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.917103 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1452/2927] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:40:13.921742 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.931858 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae055c2-df43-45ae-8378-40d2d37d0ba0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:13.937339 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:13.938008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:13.938288 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:13.938609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] PATH_INFO: `/v3/projects/e87edee0c7684af4accc4ccc697800dd/users/4021b0923b6041fd9883fe3c463097be/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:13.953694 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87edee0c7684af4accc4ccc697800dd, user_id=4021b0923b6041fd9883fe3c463097be, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:13.956395 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:13.970344 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1476/2928] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:13 2022] PUT /identity/v3/projects/e87edee0c7684af4accc4ccc697800dd/users/4021b0923b6041fd9883fe3c463097be/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:13.974219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:13.984858 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca406d-0f8e-4bde-9d7c-d55921c13e50 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:14.050973 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:14.051517 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:14.051799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:14.052074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:14.052722 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:14.054331 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c38de-cf85-4630-aaf0-d85aa6efe300 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:14.064584 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1453/2929] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:14.068883 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:14.080090 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d112d7-fd5b-4529-be93-fec872738363 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:14.086089 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:14.086814 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:14.087129 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:14.087415 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] PATH_INFO: `/v3/projects/e87edee0c7684af4accc4ccc697800dd/users/4021b0923b6041fd9883fe3c463097be/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:14.089668 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e87edee0c7684af4accc4ccc697800dd, user_id=4021b0923b6041fd9883fe3c463097be, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:14.092455 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:14.104136 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1477/2930] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:14 2022] PUT /identity/v3/projects/e87edee0c7684af4accc4ccc697800dd/users/4021b0923b6041fd9883fe3c463097be/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:14.114534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:14.114824 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:14.115115 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:14.122887 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:14.146422 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] MFA Rules not processed for user `4021b0923b6041fd9883fe3c463097be`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:14.147646 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:14.219542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1454/2931] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:14.230813 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:14.232933 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:14.291493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f5aa2-71ec-4749-84ab-ce8c28e830d6 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:14.295612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:14.296222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:14.296503 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:14.296770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:14.306267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:14.307695 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:14.317612 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1478/2932] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:14.867127 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:14.869705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:14.887326 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd51f9-5d66-4fea-88b7-db4a0765e6e0 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:14.891942 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:14.892877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:14.893357 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:14.893697 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:14.962002 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:14.963498 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:14.973783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1455/2933] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:19.060324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:19.069805 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655399fb-ab0f-4177-8b72-5d41114c0ca0 service cinder] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:19.074025 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:19.074673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:19.074957 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:19.075227 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:19.133855 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:19.135490 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0767ce9-1dd6-4247-85ea-92cb417b7620 service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:19.144607 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1479/2934] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:21.343448 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:21.406266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4df8b-778a-42ee-98fa-643d9a7b5748 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:21.411169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:21.411799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:21.412077 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:21.412350 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] PATH_INFO: `/v3/users/655c253187444b428b941157118c6a7f` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:21.413521 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] RBAC: Authorizing `identity:delete_user(user_id=655c253187444b428b941157118c6a7f)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:21.415516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:21.448098 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:21.468543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655c253187444b428b941157118c6a7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:21.476330 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655c253187444b428b941157118c6a7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:21.488483 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655c253187444b428b941157118c6a7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:21.489104 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655c253187444b428b941157118c6a7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:21.495937 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655c253187444b428b941157118c6a7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:21.501815 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1456/2935] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:21 2022] DELETE /identity/v3/users/655c253187444b428b941157118c6a7f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:21.506417 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:21.506920 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:21.507022 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:21.526206 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:21.527448 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:21.569673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:21.589851 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1480/2936] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:21.599812 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:21.601306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:21.660460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93578507-cc2c-4cd7-b4cc-4b13c9fce01f admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:21.664591 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:21.665192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:21.665473 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:21.665744 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:21.675263 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:21.676705 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:21.686638 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1457/2937] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:21.973772 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:21.975254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:21.987671 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca074dd4-b571-4589-a0d4-87906dbfda61 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:21.993059 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:21.993735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:21.994130 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:21.994427 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] PATH_INFO: `/v3/projects/4129293efd83403887c1dd3d8975969a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:21.995770 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4129293efd83403887c1dd3d8975969a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:21.997656 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:22.047242 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4129293efd83403887c1dd3d8975969a', 'request_id': 'req-7396ec85-5fe2-4fdf-aba7-604d4013e24b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.053262 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4129293efd83403887c1dd3d8975969a', 'request_id': 'req-7396ec85-5fe2-4fdf-aba7-604d4013e24b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.059514 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4129293efd83403887c1dd3d8975969a', 'request_id': 'req-7396ec85-5fe2-4fdf-aba7-604d4013e24b'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.075573 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] The token cache is being invalidate because project 4129293efd83403887c1dd3d8975969a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:22.075860 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.076511 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.078047 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1481/2938] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:21 2022] DELETE /identity/v3/projects/4129293efd83403887c1dd3d8975969a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:22.082224 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:22.091877 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b94c-da66-480c-87a2-893f6b62ee55 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:22.152546 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:22.153191 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:22.153539 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:22.153808 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] PATH_INFO: `/v3/users/b2852f3b1b8b450fabb10123f4ecd680` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:22.155023 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2852f3b1b8b450fabb10123f4ecd680)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:22.156977 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:22.206001 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2852f3b1b8b450fabb10123f4ecd680'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.208388 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:22.214512 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2852f3b1b8b450fabb10123f4ecd680'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.227369 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2852f3b1b8b450fabb10123f4ecd680'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.227996 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2852f3b1b8b450fabb10123f4ecd680'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.235045 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2852f3b1b8b450fabb10123f4ecd680'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.241356 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1458/2939] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:22 2022] DELETE /identity/v3/users/b2852f3b1b8b450fabb10123f4ecd680 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:22.515139 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:22.516696 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:22.529761 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396ec85-5fe2-4fdf-aba7-604d4013e24b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:22.590410 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:22.591067 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:22.591349 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:22.591621 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] PATH_INFO: `/v3/projects/a7a9368a9f5c4e808dac4785f8cd75a0` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:22.592767 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a9368a9f5c4e808dac4785f8cd75a0)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:22.594739 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:22.646299 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a9368a9f5c4e808dac4785f8cd75a0', 'request_id': 'req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.654397 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a9368a9f5c4e808dac4785f8cd75a0', 'request_id': 'req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.661000 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a9368a9f5c4e808dac4785f8cd75a0', 'request_id': 'req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.672041 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:22.678583 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] The token cache is being invalidate because project a7a9368a9f5c4e808dac4785f8cd75a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:22.678938 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.679721 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.682154 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1482/2940] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:22 2022] DELETE /identity/v3/projects/a7a9368a9f5c4e808dac4785f8cd75a0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:22.686861 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:22.688509 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:22.701760 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fdfaa-00a6-42c3-91a6-a14a7a9b7f4d admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:22.762167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:22.762839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:22.763122 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:22.763395 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] PATH_INFO: `/v3/users/4021b0923b6041fd9883fe3c463097be` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:22.764549 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4021b0923b6041fd9883fe3c463097be)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:22.766518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:22.821286 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4021b0923b6041fd9883fe3c463097be'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.830323 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4021b0923b6041fd9883fe3c463097be'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.847364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4021b0923b6041fd9883fe3c463097be'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.848120 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4021b0923b6041fd9883fe3c463097be'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.856051 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a380abce-ff20-4797-880f-be72ee16e472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4021b0923b6041fd9883fe3c463097be'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:22.862151 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1459/2941] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:22 2022] DELETE /identity/v3/users/4021b0923b6041fd9883fe3c463097be => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:23.141084 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.142697 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:23.155593 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0a2d7-ae2e-49ea-9725-df4cd4d46eea admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.216457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.217111 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.217390 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.217673 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] PATH_INFO: `/v3/projects/e87edee0c7684af4accc4ccc697800dd` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.218968 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e87edee0c7684af4accc4ccc697800dd)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.220909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.269844 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e87edee0c7684af4accc4ccc697800dd', 'request_id': 'req-1a7f75c2-da0d-47c6-9c91-210c7a305e23'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:23.275893 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e87edee0c7684af4accc4ccc697800dd', 'request_id': 'req-1a7f75c2-da0d-47c6-9c91-210c7a305e23'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:23.281985 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e87edee0c7684af4accc4ccc697800dd', 'request_id': 'req-1a7f75c2-da0d-47c6-9c91-210c7a305e23'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:23.295676 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] The token cache is being invalidate because project e87edee0c7684af4accc4ccc697800dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:23.295964 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:23.296596 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-1a7f75c2-da0d-47c6-9c91-210c7a305e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:23.298189 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1483/2942] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:23 2022] DELETE /identity/v3/projects/e87edee0c7684af4accc4ccc697800dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:23.304254 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.304534 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.304804 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.325819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:23.327074 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:23.370723 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:23.390730 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1460/2943] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:23.399358 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.399645 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.399916 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.420336 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:23.421545 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:23.430167 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1484/2944] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:23.434856 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.445235 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9249f-7b79-4301-a59b-b0f131a00b26 None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.451073 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.451675 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.451950 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.452215 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.452868 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:40:23.453167 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.454810 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.466571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1461/2945] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Sep 13 20:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:40:23.471764 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.481318 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eed2b-f942-49bb-8304-ba07d8cca56f None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.487392 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.488028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.488300 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.488572 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.489217 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.491183 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.523101 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1485/2946] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:40:23.527336 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.537508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676b991-c0ff-4504-ad1f-ab950c401fa6 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.596469 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.597085 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.597366 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.597635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.598261 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.600430 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.625647 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1462/2947] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:23.632231 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.643875 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43fe67-9116-4322-b6b5-2311f8de9ce8 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.648926 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.649537 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.649817 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.650088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.650768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.652394 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.662899 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1486/2948] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:23.667216 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.683497 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ccc176-e4b5-4349-bb68-e316f752a753 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.691915 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.692917 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.693288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.693717 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.694820 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.697412 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3bf67-b70f-4c26-9297-65e46eb919fb admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.709628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1463/2949] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:40:23.713722 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.723869 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d42d-1efb-411c-9d96-4ef57829e974 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:23.729881 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:23.730617 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.730972 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.731241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] PATH_INFO: `/v3/projects/32e16f22296944179c5b6ae63a4b3639/users/98df7016f55d4772bf0a052722dc6a59/roles/753c4dc548cd4aeabce829afb96fed52` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.747133 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e16f22296944179c5b6ae63a4b3639, user_id=98df7016f55d4772bf0a052722dc6a59, role_id=753c4dc548cd4aeabce829afb96fed52)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:23.749799 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:23.765567 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1487/2950] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:23 2022] PUT /identity/v3/projects/32e16f22296944179c5b6ae63a4b3639/users/98df7016f55d4772bf0a052722dc6a59/roles/753c4dc548cd4aeabce829afb96fed52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:40:23.844806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:23.845209 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:23.845692 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:23.879783 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.auth.core [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] MFA Rules not processed for user `98df7016f55d4772bf0a052722dc6a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:23.881008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:23.953556 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1464/2951] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:23.958501 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:23.986716 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:24.037246 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48e9be-2540-4896-ad86-21a12bf69a81 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.042149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.042818 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.043102 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.043380 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.044008 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.045960 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.074457 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1488/2952] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Sep 13 20:40:23 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:24.078586 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.089773 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4ea90-124a-4d2b-9175-871a30c5d31e None None] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.131680 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:24.150437 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.151059 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.151340 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.151612 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.152251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.154303 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.177301 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1465/2953] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Sep 13 20:40:24 2022] POST /identity/v3/users => generated 307 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:40:24.181400 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.192618 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e150-9b46-4946-af91-9ef819743482 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.197372 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.197983 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.198263 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.198578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.199267 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.200909 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.211708 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1489/2954] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:40:24.215799 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.226180 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cff34-1bc6-43d6-9cef-450e87227a45 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.231306 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.231940 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.232367 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.232682 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] PATH_INFO: `/v3/projects/5dc90b108e4b443ba32e83285e385788/users/41ee8ac420e343d2a648dba30ed7c91c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.248189 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc90b108e4b443ba32e83285e385788, user_id=41ee8ac420e343d2a648dba30ed7c91c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.250880 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.264311 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1466/2955] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:24 2022] PUT /identity/v3/projects/5dc90b108e4b443ba32e83285e385788/users/41ee8ac420e343d2a648dba30ed7c91c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:40:24.268366 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.278212 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4205fe-214e-4bff-8ca0-ab501aab2a6b admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.335923 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.336609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.336884 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.337149 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.337790 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.339436 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7dc76-21b7-4c79-bdcd-52bbbbb77efe admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.349837 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1490/2956] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Sep 13 20:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:40:24.354065 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.364894 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91a0df-26a6-400b-82f1-1a7b90931523 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.370571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.371230 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.371508 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.371774 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] PATH_INFO: `/v3/projects/5dc90b108e4b443ba32e83285e385788/users/41ee8ac420e343d2a648dba30ed7c91c/roles/97fb86a8759e4f148719d1d705a0d44c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.373932 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc90b108e4b443ba32e83285e385788, user_id=41ee8ac420e343d2a648dba30ed7c91c, role_id=97fb86a8759e4f148719d1d705a0d44c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.376628 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.387434 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1467/2957] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Sep 13 20:40:24 2022] PUT /identity/v3/projects/5dc90b108e4b443ba32e83285e385788/users/41ee8ac420e343d2a648dba30ed7c91c/roles/97fb86a8759e4f148719d1d705a0d44c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:40:24.397977 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.398343 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.398768 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.434146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] MFA Rules not processed for user `41ee8ac420e343d2a648dba30ed7c91c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:24.435406 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:24.504338 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1491/2958] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:24.515456 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.517453 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:24.578046 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9ebaa-c7e9-4fe0-8b9c-9c0e2e50c6fc admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.582027 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.582672 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.582949 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.583219 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.592460 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.593891 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.603364 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1468/2959] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:24.694493 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:24.704987 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f3e99-1762-443e-bea1-094eb5d75c1d None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:24.708898 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f641d3de0a3a43ad8bc789483ffd5a72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:24.709505 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:24.709783 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:24.710050 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:24.726902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:24.766906 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:24.768329 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d2b90-0f10-49f5-8a14-e8369a5c5bbe service cinder] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:24.777522 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1492/2960] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:28.961518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:29.025707 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc53a5-3d2c-44f4-8248-264b34e637d0 service cinder] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:29.030997 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:29.031636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:29.032809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:29.032809 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] PATH_INFO: `/v3/users/98df7016f55d4772bf0a052722dc6a59` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:29.033390 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98df7016f55d4772bf0a052722dc6a59)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:29.035389 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:29.092572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:29.100635 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98df7016f55d4772bf0a052722dc6a59'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.108736 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98df7016f55d4772bf0a052722dc6a59'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.122995 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98df7016f55d4772bf0a052722dc6a59'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.124082 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98df7016f55d4772bf0a052722dc6a59'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.132762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98df7016f55d4772bf0a052722dc6a59'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.139307 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1469/2961] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:28 2022] DELETE /identity/v3/users/98df7016f55d4772bf0a052722dc6a59 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:29.144874 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] REQUEST_METHOD: `POST` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:29.145701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:29.145701 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:29.171230 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.auth.core [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] MFA Rules not processed for user `0f81252e23c341beb35dc2c378ed375c`. Rule list: `[]` (Enabled: `True`). {{(pid=81575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:40:29.172437 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:29.236940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1493/2962] 10.222.0.52 () {58 vars in 958 bytes} [Tue Sep 13 20:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:40:29.246960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:29.248518 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:29.306285 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261637f-a763-421d-b756-2a6d01a54ef2 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:29.310225 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:29.311226 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:29.311516 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:29.311780 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:29.321199 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:29.322674 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:29.332251 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1470/2963] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:29.628608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:29.630104 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:29.645446 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555558-990e-4391-876b-bae4861bd7f0 None None] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:29.650471 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:29.651244 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:29.651528 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:29.651812 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] PATH_INFO: `/v3/projects/32e16f22296944179c5b6ae63a4b3639` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:29.653141 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e16f22296944179c5b6ae63a4b3639)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:29.655258 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:29.704838 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e16f22296944179c5b6ae63a4b3639', 'request_id': 'req-00fd3ed8-16e9-46e1-8847-f701f0a615ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.710608 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e16f22296944179c5b6ae63a4b3639', 'request_id': 'req-00fd3ed8-16e9-46e1-8847-f701f0a615ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.716811 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e16f22296944179c5b6ae63a4b3639', 'request_id': 'req-00fd3ed8-16e9-46e1-8847-f701f0a615ca'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.730824 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] The token cache is being invalidate because project 32e16f22296944179c5b6ae63a4b3639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:29.731112 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.731736 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.733438 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1494/2964] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:29 2022] DELETE /identity/v3/projects/32e16f22296944179c5b6ae63a4b3639 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:29.737288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:29.747383 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b9eb4-cacb-495f-aec7-e6e542b36d0f service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:29.804561 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:29.805200 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:29.805481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:29.805762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] PATH_INFO: `/v3/users/41ee8ac420e343d2a648dba30ed7c91c` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:29.806980 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ee8ac420e343d2a648dba30ed7c91c)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:29.808909 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:29.857333 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ee8ac420e343d2a648dba30ed7c91c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.865318 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ee8ac420e343d2a648dba30ed7c91c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.870826 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:29.879320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ee8ac420e343d2a648dba30ed7c91c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.880247 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ee8ac420e343d2a648dba30ed7c91c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.887320 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ee8ac420e343d2a648dba30ed7c91c'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:29.893763 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1471/2965] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:40:29 2022] DELETE /identity/v3/users/41ee8ac420e343d2a648dba30ed7c91c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:30.179073 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:30.180589 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:30.194549 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd3ed8-16e9-46e1-8847-f701f0a615ca admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:30.239088 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:30.257484 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:30.258158 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:30.258578 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:30.258863 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] PATH_INFO: `/v3/projects/5dc90b108e4b443ba32e83285e385788` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:30.259980 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc90b108e4b443ba32e83285e385788)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:30.261907 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:30.309319 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc90b108e4b443ba32e83285e385788', 'request_id': 'req-31710722-57d6-4a01-b773-9453095aadf0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:30.315609 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc90b108e4b443ba32e83285e385788', 'request_id': 'req-31710722-57d6-4a01-b773-9453095aadf0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:30.321894 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc90b108e4b443ba32e83285e385788', 'request_id': 'req-31710722-57d6-4a01-b773-9453095aadf0'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:30.336241 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] The token cache is being invalidate because project 5dc90b108e4b443ba32e83285e385788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:40:30.336566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:30.337254 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:40:30.339280 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1495/2966] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:40:30 2022] DELETE /identity/v3/projects/5dc90b108e4b443ba32e83285e385788 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:40:39.890324 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:40.332940 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:56.154888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:56.156747 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:56.218627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51013bac-06a3-4d93-81c1-a455f55a6ddd admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:56.267165 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:56.267781 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:56.268061 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:56.268334 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:56.270511 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:56.280806 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:56.282249 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:56.290994 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1472/2967] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:06.287092 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=46 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:22.944888 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:22.946760 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:22.961946 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710722-57d6-4a01-b773-9453095aadf0 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:22.967011 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:22.967712 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:22.967965 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:22.968309 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:22.970558 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:23.038015 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:23.039491 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:23.048840 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1496/2968] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:41:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:24.142384 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:24.145481 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:24.157574 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b51ece-f919-43c3-b14a-46b5a690fe08 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:24.161551 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:24.162169 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:24.162487 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:24.162762 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:24.164708 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:24.238904 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:24.240552 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:24.252839 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1473/2969] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:41:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:25.442704 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:25.451902 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cbb5c-88fb-4587-b4c2-f6783c791bb2 service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:25.456637 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:25.457245 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:25.457521 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:25.457795 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:25.460109 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:25.479468 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:25.481669 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:25.494236 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1497/2970] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:41:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:34.247757 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=34 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:35.489147 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=34 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:36.620543 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:36.629931 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0175-6328-4e43-813d-696d39c162c3 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:36.634500 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a83958d50934b24ae40da8760b2fb85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:36.635110 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:36.635387 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:36.635655 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:36.637590 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:36.703827 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:36.705243 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:36.714772 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1474/2971] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:39.822822 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:39.826566 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:39.930715 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0469df0-02c8-4e4f-888d-cb7e06fbceed service neutron] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:39.937142 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:39.937765 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] REQUEST_METHOD: `GET` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:39.938049 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:39.938321 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:39.940162 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:39.952093 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:39.953698 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:39.964445 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1498/2972] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:41:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:41:43.002681 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:43.005960 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:43.031656 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789007e-88d0-46c8-abfd-e9289d47fae3 service neutron] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:43.042830 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:43.043570 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:43.043699 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:43.044014 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] PATH_INFO: `/v3/users/d20ba942c80848d7819aede8c0df1e7f` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:43.045192 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20ba942c80848d7819aede8c0df1e7f)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:43.047172 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:43.093954 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20ba942c80848d7819aede8c0df1e7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.102332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20ba942c80848d7819aede8c0df1e7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.114971 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20ba942c80848d7819aede8c0df1e7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.115701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20ba942c80848d7819aede8c0df1e7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.122701 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20ba942c80848d7819aede8c0df1e7f'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.128821 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1475/2973] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:41:43 2022] DELETE /identity/v3/users/d20ba942c80848d7819aede8c0df1e7f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:41:43.305850 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:43.319672 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb943c-355b-4d28-84ea-9492114a7a2f service placement] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:43.380918 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:43.381611 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:43.381866 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:43.382135 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] PATH_INFO: `/v3/projects/7878c5e6e1e143199681f42b5518cc1a` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:43.383314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7878c5e6e1e143199681f42b5518cc1a)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:43.385222 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:43.438735 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7878c5e6e1e143199681f42b5518cc1a', 'request_id': 'req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.445519 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7878c5e6e1e143199681f42b5518cc1a', 'request_id': 'req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.451719 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7878c5e6e1e143199681f42b5518cc1a', 'request_id': 'req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.464743 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] The token cache is being invalidate because project 7878c5e6e1e143199681f42b5518cc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:41:43.465156 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.465776 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.467353 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1499/2974] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:41:43 2022] DELETE /identity/v3/projects/7878c5e6e1e143199681f42b5518cc1a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:41:43.471627 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:43.473181 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:43.486405 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562773-c675-4cc9-85cc-de9cea921446 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:43.544332 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:43.544990 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:43.545270 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:43.545542 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] PATH_INFO: `/v3/users/89c43023f8974882bf7ed6268d62053d` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:43.546923 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c43023f8974882bf7ed6268d62053d)` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:43.548965 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:43.555585 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:43.597888 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c43023f8974882bf7ed6268d62053d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.606572 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c43023f8974882bf7ed6268d62053d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.619819 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c43023f8974882bf7ed6268d62053d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.620563 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c43023f8974882bf7ed6268d62053d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.627822 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.notifications [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c43023f8974882bf7ed6268d62053d'} {{(pid=81576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.634288 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1476/2975] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Sep 13 20:41:43 2022] DELETE /identity/v3/users/89c43023f8974882bf7ed6268d62053d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:41:43.818314 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Authenticating user token {{(pid=81575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:41:43.819976 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.fernet_utils [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:41:43.833347 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ab3c-22ea-4b46-9b83-ae57c2f74c89 admin admin] Validating token access rules against request {{(pid=81575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:41:43.852192 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:43.894934 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f81252e23c341beb35dc2c378ed375c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02e765af7d454a8f9fe205c67fe1fdbb', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:41:43.895582 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:41:43.895862 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:41:43.896132 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] PATH_INFO: `/v3/projects/4ece4b2e7bbc45dfb841474508551daa` {{(pid=81575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:41:43.897303 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ece4b2e7bbc45dfb841474508551daa)` {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:41:43.899324 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] RBAC: Authorization granted {{(pid=81575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:41:43.951773 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ece4b2e7bbc45dfb841474508551daa', 'request_id': 'req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.959297 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ece4b2e7bbc45dfb841474508551daa', 'request_id': 'req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.968146 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ece4b2e7bbc45dfb841474508551daa', 'request_id': 'req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7'} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.984796 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] The token cache is being invalidate because project 4ece4b2e7bbc45dfb841474508551daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:41:43.985094 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.985710 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG keystone.notifications [None req-5a38c8fb-1334-44ee-bcc4-0e5a5d4d66e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:41:43.987233 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: [pid: 81575|app: 0|req: 1500/2976] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Sep 13 20:41:43 2022] DELETE /identity/v3/projects/4ece4b2e7bbc45dfb841474508551daa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:41:53.630636 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:41:53.982028 dv-cinder-854274-5-iscsi devstack@keystone.service[81575]: DEBUG dbcounter [-] [81575] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:42:51.385704 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Authenticating user token {{(pid=81576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:42:51.387248 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:42:51.457008 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7c18d-cda3-4c81-bc41-4721e3d10397 admin admin] Validating token access rules against request {{(pid=81576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:42:51.521351 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7bec1fe5f26425eb9d3e98f7502a7d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5597c130581c48e7bcd2fdd79c5e72e0', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:42:51.521991 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] REQUEST_METHOD: `GET` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:42:51.522266 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] SCRIPT_NAME: `/identity` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:42:51.522571 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:42:51.524484 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.fernet_utils [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:42:51.536445 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:42:51.538036 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2bdb2-54f7-4772-b868-5609ecbdaae3 service placement] RBAC: Authorization granted {{(pid=81576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:42:51.548535 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: [pid: 81576|app: 0|req: 1477/2977] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Sep 13 20:42:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:43:01.544900 dv-cinder-854274-5-iscsi devstack@keystone.service[81576]: DEBUG dbcounter [-] [81576] Writing DB stats keystone:SELECT=58 {{(pid=81576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}